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ABSTRACT 


For decades, the United States’ major defense acquisition system has been under 
scrutiny and undergone much reform. Groups have researched the issues, publishing 
hundreds of reports identifying various problems and solutions. Yet, many major weapon 
systems continue to be well over budget and schedule. Major weapon systems are 
increasing in size, scope, and complexity. Technology is rapidly changing. Customer 
expectations are rising. Societal concerns, such as workforce and economic 
development, are playing a bigger role. Politics are rampant in this system. This system 
qualifies as a CLIOS system—Complex, Large-scale, Interconnected, Open, and 
Sociotechnical in nature. 

This thesis explored and analyzed the decades of research concerning U.S. major 
weapon systems acquisitions and applied the CLIOS Process. Of the three stages within 
the CLIOS Process, this research applied the Representation Stage to the U.S. major 
defense acquisition system. The observations afforded from the analysis were: (1) long¬ 
term decisions are made with short-term information and (2) multiple stakeholders and 
decision makers facilitate little accountability. Three strategic alternatives were 
identified: (1) create an Integrated Process Team to make joint long-term decisions, (2) 
mandate a Federal Systems Engineering organization, and (3) create a hybrid between 
the first two for instilling accountability at all levels. 
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EXECUTIVE SUMMARY 


For decades, the United States’ major defense acquisition system has been under 
scrutiny and undergone much reform. Numerous groups have researched the issues, 
publishing hundreds of reports identifying various problems and solutions. Yet, many 
major weapon systems continue to be well over budget and schedule. 

The U.S. major defense acquisition system is increasing in size, scope, and 
complexity each year. Technology associated with major weapon systems is rapidly 
changing. Customer expectations are rising. Societal concerns, such as workforce and 
economic development, are playing an increasingly bigger role. Politics continue to play 
a part in this system. This system certainly qualifies as a CLIOS system—Complex, 
Large-scale, Interconnected, Open, and Sociotechnical in nature. 

This thesis explored and analyzed the decades of research concerning U.S. major 
weapon systems acquisitions. The research introduced the CLIOS Process developed by 
MIT to analyze and study complex systems. The basic structure of the CLIOS Process is 
three stages encompassing 12 steps. The three stages include (1) Representation; (2) 
Design, Evaluation, and Selection; and (3) Implementation. 

Of the three stages within the CLIOS Process, this research applied the 
Representation stage to the U.S. major defense acquisition system. This exploratory 
approach described the system as a whole, identified subsystems, and described the 
components and links among the subsystems and major actor groups. Despite continuing 
with the CLIOS Process, initial analysis was conducted based on the observations of 
Stage 1. This resulted in two observations: (1) Long-term decisions made on short-term 
information and (2) Independent working leads to lesser accountability. Three strategic 
alternatives were identified: (1) form an Integrated Process Team with Congress, JCIDS 
and the Defense Acquisition System to jointly consider both near and far-term economic 
conditions and security threats before new long-term defense programs were approved 
and funded; (2) form a mandated Federal Systems Engineering organization equally 
accountable to both Congress and DoD to consider economic and threat conditions, 

xiii 



analyze technological maturity, and make independent cost estimates as inputs to new 
defense system acquisition decisions; and (3) create a hybrid of the two previous 
solutions to implement accountability at all levels of the acquisition to all major actor 
groups. Of course, challenges to implementation will exist. However, continuing with 
the CLIOS Process will facilitate in refining the strategic alternatives, as well as 
implementing them. 

Further research should be conducted on the U.S. major defense acquisition 
system using the CLIOS Process. The remaining stages of the CLIOS Process, especially 
the iterative aspects of the process, should be applied to the U.S. major defense 
acquisition system. 
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I. INTRODUCTION 


A, BACKGROUND 

For more than a half century, issues surrounding the United States’ major 
weapons system acquisition have been scrutinized and discussed. Watchdog agencies 
have researched the issues and made recommendations, publishing hundreds of reports 
identifying numerous problems and solutions. While improvements and reforms have 
been made, many major weapon systems continue to suffer from requirements creep, 
budget and schedule overruns, performance issues, and much more. According to an 
April 2009 report published by the United States Government Accountability Office on 
major weapon system acquisition, the Department of Defense’s 2008 portfolio of major 
defense acquisition programs is already showing signs of trouble. Total research and 
development costs are on average 42 percent higher than the first estimate. Total 
acquisition costs are 25 percent higher than the first estimate. The average delay in 
delivering initial capabilities is 22 months (GAO, 2009). 

The system for building and acquiring U.S. major weapon systems can be 
classified as a “CLIOS” system—Complex, Large-scale, Interconnected, Open, and 
Sociotechnical in nature (Sussman, 2007). This system of acquiring major weapons 
proves to be more complex and dynamic than ever imagined: 

. There is rapidly changing technology. 

. The time between concept design to delivery can take decades. 

. There are social, political, and economic factors intertwined in the 
process. 

. The end user is not the customer. 

• The government has a need to retain specialized skills, yet has a limited 
number of programs on which to bid and “keep” defense contractors 
active while still meeting requirements. 
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• The U.S. defense industrial base is exclusively private-owned and thus 
governed by independent business decisions. 

. The “military industrial complex” is both a burden and necessity to the 
U.S. government. 

The U.S. major defense acquisition system is a very complex system with sub¬ 
systems and many internal and external influences. Much has been written and 
documented on this process. Repeatedly, groups have highlighted the immediate need of 
meeting planned schedules and budgets. Yet, there is little if any “improvement” on the 
issues that have plagued this industry and process for decades. The focal point of past 
research and publications concentrated on immediate solutions. Although solutions are 
necessary, the same issues and recommendations are given year after year. 

In order to effectively address issues within this giant, complex system, it first 
must be treated as such. The U.S. major defense acquisition is increasing in size, scope, 
and complexity each year. The technology associated with major weapon systems is 
rapidly changing. Customer expectations are rising. Societal concerns such as workforce 
and economic development are playing an increasingly larger role. Politics continue to 
play a part in this system. The U.S. major defense acquisition system certainly qualifies 
as a CLIOS system—Complex, Large-scale, Interconnected, Open, and Sociotechnical in 
nature. 

B, PURPOSE AND BENEFITS OF STUDY 

The purpose of this study is to gain a better understanding of the system of 
acquiring major weapon systems as a whole, identify the emergent behavior of this 
system, find possible points of leverage for improvements to the system, and begin 
applying the CLIOS framework for a different approach and understanding of the system. 

The goal is twofold; exploratory and descriptive. The study will explore previous 
research over the past decades concerning major weapon systems acquisition. Many of 
the findings outline the immediate needs of programs’ ability to meet the set schedule and 
budget. The purpose of exploring the literature is to better understand how the system 
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has behaved in the past, understand the areas of eoneern, and identify the pieees and 
players within the system. This study will offer an overview of signifieant researeh and 
literature published about major weapon systems aequisition. 

This study will also deseribe the major defense aequisition system in order to 
ensure a holistie view is eaptured. Arehiteeting the system will aid in the understanding 
of all of the systems at play within the major defense aequisition system as well as the 
interfaees. 

C. RESEARCH QUESTIONS 

This thesis will seek to answer the following researeh questions: 

What are the issues identified in previous literature eoneeming U.S. major 
weapon systems aequisition? 

How does the United States’ system of aequiring major weapons eompare to other 
top military spending nations? 

What are the pieees, and/or who are the players, within the major defense 
aequisition system? 

What other areas of the major defense aequisition system should be identified and 
researehed when applying the CLIOS proeess? 

Will the CLIOS proeess provide more understanding of the issues and enable 
solutions to the U.S major defense aequisition proeess that have not been previously 
reviewed or suggested? 

D, METHODOLOGY 

This exploratory and deseriptive researeh is a ease study of the major defense 
aequisition system. In order to effeetively address issues within this giant, eomplex 
system, it first must be treated as sueh. The U.S. major defense aequisition proeess is 
only inereasing in size, seope, and eomplexity with eaeh year and eertainly qualifies as a 
CLIOS system—Complex, Large-seale, Intereonneeted, Open, and Soeio-teehnieal in 


nature. 
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Chapter II will provide a comparison of the U.S. defense acquisition system to 
acquisition systems of other top military spending countries. It will end with a review of 
the major defense acquisition literature. This literature review will focus on the critical 
points of past and current knowledge of the issues surrounding the U.S. system of major 
weapon systems acquisition. This review will take a step back, explore, and analyze the 
research and publications, which identified issues and solutions towards U.S. major 
weapon systems acquisitions. 

This exploratory approach will seek to identify gaps in the understanding of the 
system as a whole using the CLIOS Process as a framework. The CLIOS Process was 
created by Joseph Sussman, a professor in the Engineering Systems Division Program at 
the Massachusetts Institute of Technology (MIT), in order to “describe, understand, 
study, and, ultimately, to improve the performance of a...system...with wide-ranging 
economic, social, political, and environmental impacts” (Sussman, 2007, p. 6). This 
process will be introduced and reviewed in Chapter III of this study. 

Chapter IV will present a partial CLIOS representation of the major defense 
acquisition system, which is an approach to architecting the system. This allows for 
describing the system, identifying subsystems, and describing components and links. 
Finally, Chapter V will conclude the study with key points and recommendations for 
improvements and further research on the system of acquiring major weapon systems. 
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II. REVIEW OF LITERATURE 


A, INTRODUCTION 

“It is clear that the problems being encountered today are similar to those of the 
past.” (GAO, 1988) 

Issues of inereased eosts, schedule delays, and degradation of performanee 
requirements have plagued the United States’ major weapons aequisition system for more 
than half a eentury. Scholars, researchers, and practitioners alike have studied the 
aequisition of major weapon systems. The pure quantity of reports generated makes it 
infeasible to review all of them. While a plethora of literature assoeiated with this subject 
exists, only key reports have been chosen to provide a broad understanding of the issues, 
reasons and reeommendations, and change initiatives provided by these researehers. 

B, ISSSUES SURROUNDING MAJOR WEAPON SYSTEMS ACQUISITION 

Marshall and Meekling (1959) eondueted researeh on 22 major weapon systems 
to better prediet the sueeess of development and produetion. They identified three faetors 
impeding the sueeess of new programs: eost, time, and performance. Marshall and 
Meekling discovered eost inereases of 200-300%. The aetual eosts for missiles were, on 
average, 5.1 times greater than the original estimates. On average, the actual costs for 
bombers were 3.1 times greater than the original estimates. Coneerning time, Marshall 
and Meekling found data for only ten of the 22 systems they researehed, where time 
extensions of 33-50% were the norm for newer weapon systems aequisitions. Of those 
ten systems, the average “slippage” from original estimate to aetual availability was two 
years. Although performanee was more difficult to quantity (such as the use of dollars 
and years for the previous two factors), Marshall and Meekling found that most of the 22 
systems “fell short of performanee expeetations” (p. 21). 

GAO (1969, November) reported on the Army’s purehase of Sheridan weapon 
systems, M60A1E1 tank turrets, and M60A1E2 tanks. The tanks did not meet 
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performance expectations nor schedule deadlines, with time and performance being two 
of the factors identified in Marshall and Meckling’s research. Basically, the tanks were 
rapidly produced before adequate ammunition was created and tested. However, the 
acquisition of tanks continued, ultimately forcing them to be stored unused. 

In 1969, Congress asked GAO to periodically report on the status of various 
major weapon systems. Fifty-seven programs were selected for the status on the 
acquisition of major weapon systems in February 1970 (GAO, 1970). However, only 38 
had sufficient data for comparison. Initial cost estimates were significantly misjudged 
with estimates of program completion 50% higher than original estimates. Along with 
cost, performance was an issue on many of the programs. A large number of weapon 
systems had considerable variance in performance from the original estimate to the 
subsequent estimates before production. GAO also reported schedule delays on most 
programs ranging from six months to three years. 

In June 1971, Perry, Smith, Harman, and Henrichsen published the findings of a 
two-year study on 24 major systems developed during the 1960s. The focus of these 
Rand researchers was the three factors identified by Marshall and Meckling: cost, 
schedule, and performance. Data for the three factors on all 24 systems was not 
available. However, Perry et al. were able to gather significant data as well as research 
causes and possible solutions. The cost data revealed an average increase of 40%, while 
schedule delays averaged 15% (Perry et al., 1971). 

The GAO released a comprehensive review of selected Air Force, Navy, and 
Army major weapon systems for the 1971 status report (GAO, 1971, March). Estimated 
costs for 61 systems increased by $33.4 billion. In the 1972 status, GAO reviewed 47 
other systems that had a net increase of only $2,287.6 million during 1971 (GAO, 1972). 
Between June 30, 1972 and December 31, 1972, a selection of 45 major weapon systems 
increased in cost by $585.7 million (GAO, 1973). For six months during 1973, there was 
a net increase of $7,016.1 million on the 55 major weapon systems under review (GAO, 
1974). The 50 major weapon systems selected in 1974 saw a net increase of $13,242.5 
million (GAO, 1975). Of those 50 major weapon systems, 22 were either 12 months or 

more behind schedule. Surprisingly, the GAO status report published in 1976 showed a 
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net decrease of $2,355.9 million for the 47 major weapon systems analyzed January 1, 
1975 to June 30, 1975 (GAO, 1976). GAO continued to release reports concerning 
increased costs on selected major weapon systems throughout the 1970s. See Table 1. 


Table 1. GAO Status on Selected Major Weapon Systems 


GAO Status on Selected Major 

Weapon Systems 


Net Increase / 


# of 


Decrease from 

Year 

Weapn 

Year of Data 

Original Estimate 

Published 

systems 

1970 

$33.4 billion 


61 

1971 

$2.3 billion 

mm 

47 

1972 

$0.59 billion 


45 

1973 

$7.0 billion 

mm 

55 

1974 

$13.2 billion 

HQ 

50 

1975 

($2.4 billion) 


47 

1976 

$14.4 billion 

mm 

52 

1977 

(no data) 

HQ 


1978 

$29 billion 


60 

1979 

$97.0 billion 

mm 

58 


In his statement to congress in 1979, Jerome H. Stolarow, the Director of 
Procurement and Systems Acquisition, spoke to the lingering problem of cost growth 
saying, “Despite the level of anxiety, the publicity, and the many learned studies, the 
problem persists” (GAO, 1979, June, p. 2). 

Certainly cost growth received much attention in the 1970s. However, after 
thoroughly reviewing 21 major weapon systems, GAO released a report in 1980, which 
identified and grouped issues into 17 categories; (1) operational or performance 
limitations, (2) survivability or vulnerability, (3) availability, (4) operational 
requirements, (5) reliability, (6) force mix requirements, (7) force capabilities, (8) 
operational utility, (9) affordability, (10) data reporting incomplete, (11) program 
concurrency, (12) adequacy of testing, (13) cost effectiveness, (14) program 
management, (15) development strategy, (16) system urgency, and (17) technical risk or 
problems. These 17 categories fell into a broader category of either mission effectiveness 
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or program acquisition. Of the 75 issues found, 59% fell into the mission effeetiveness 
eategory and 41% in the program aequisition (GAO, 1980). 

A similar report was again issued in 1982. GAO seleeted 24 major weapon 
systems to analyze and found 71 issues that were grouped into 15 eategories falling into 
one of two broad areas identified (GAO, 1982). The issues were identified as either a 
system effeetiveness issue or a program aequisition issue. System effeetiveness 
aeeounted for 37% of the issues, and program aequisition aeeounted for 63% of the 
issues. 

In a statement before the Senate in 1984, Charles Bowsher, eomptroller general of 
the United States, validated the trends in both eost growth and extended sehedules over 
the past deeade among major weapon systems and expressed eoneem at this eontinued 
problem, seemingly impossible to eontrol. Bowsher emphasized the eurrent seriousness 
of eost growth on major weapon systems despite the reeognition it had reeeived for 
deeades (GAO, 1984). Four years later, GAO (1988) released another report 
underseoring the eontinued and reeurring problems with eost growth, sehedule delays, 
and defieieneies in performanee for major weapon system aequisitions over the past 18 
years. 

Despite the reviews and publieity in the previous deeades, major weapon systems 
aequisitions eontinued to have problems throughout the 1990s. In a report released 
Deeember 1992, GAO listed the major persistent issues reeurring from program to 
program, year to year; (1) Systems being aequired that may not be the most eost- 
effeetive solution to the mission need; (2) Overly optimistie eost and sehedule estimates 
leading to program instability and eost inereases; (3) Programs that eannot be exeeuted as 
planned with available funds; (4) Program aequisition strategies that are unreasonable or 
risky at best; (5) Too mueh being spent before a program is shown to be suitable for 
produetion and fielding; and (6) Individuals seeking to improperly influenee the outeome 
of the eontraeting proeess (GAO, 1992, p. 8). 

A 1998 report released by GAO highlighted DoD’s praetiee of allowing 
teehnology development to eontinue into produet development rather than having known 
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technology when production begins (GAO, 1998). Programs continued with much less 
knowledge and, thus, much higher risks. This uncertainty created production problems, 
increased costs, and schedule delays. Programs typically continued without the needed 
technology developed. In fact, many programs overpromised the technical performance 
without true knowledge of the undeveloped systems. These unknowns were not 
classified as high risks, and therefore, did not receive the needed attention in earlier 
stages of development (GAO, 1998). 

By 2005, GAO (now the United States Government Accountability Office) had a 
portfolio of major weapon systems valued at $1.3 trillion (GAO, 2005). In her testimony 
before the Senate, Katherine Schinasi, Managing Director, Acquisition and Sourcing 
Management, repeated the issues of cost growth, schedule delays, quantity reduction, and 
reduced performance (GAO, 2005). Schinasi reported the acquisition process was not 
responsive to current needs and the United States was on an “unsustainable fiscal path” 
that “will gradually erode...our economy, our standard of living, and ultimately our 
national security” (GAO, 2005, p. 3). GAO’s major weapon systems reviews revealed 
that programs with demonstrated or mature advanced technology had costs increases of 
less than one percent per unit cost. Whereas, those programs that began development 
with unproven technology had increased costs of approximately 21% over the first full 
estimate (GAO, 2005). 

At the end of 2005, the Assessment Panel of the Defense Acquisition 
Performance Assessment Project released a report identifying over 42 issue areas within 
major weapon systems acquisition (DAPAP, 2005). Fifteen issue areas were stressed as 
key, going beyond cost, schedule, and performance; program structure, acquisition 
strategy, complex acquisition system, requirements process, joint requirements 
development, need for leadership, process discipline, oversight, requirements allocation, 
program management (PM) expertise, acquisition career path, industry motivation and 
behavior, and the Planning, Programming, Budgeting, and Execution (PPBE) process. 

Again, in 2006, GAO issued a report detailing how continued recommendations 
and changes in policy have not rendered different results within major weapon systems 
acquisition (GAO, 2006). 
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c. 


REASONS AND RECOMMENDATIONS 


Marshall and Meckling’s (1959) conclusions were (1) early estimates were 
inaeeurate, partly due to overoptimism and (2) estimates became more aeeurate as the 
system progressed. From their researeh, Marshall and Meekling diseovered the penalties 
for underestimating cost and times were minimal compared to the opportunity for 
contractors to obtain the contraet. Also, there was much uncertainty surrounding 
technological advances, hence the estimates becoming more accurate as the systems 
progressed. Marshall and Meekling (1959) noted the inaceuracies in cost, availability, 
and performance greatly affected military and procurement decisions. 

In 1969, Elmer Staats, Comptroller General of the United States, gave a statement 
before the Subcommittee on Economy in Government, Joint Economie Committee on the 
Military Budget and National Economic Priorities (GAO, 1969, June). In his statement, 
Mr. Staats confirmed there was no standard procedure for contractors to adequately 
identify problems on major weapons programs coneeming cost, schedule, and technical 
performance. Mr. Staats eontinued to confirm one of the greatest problems with major 
weapon systems aequisition was understanding how ehanges made to contracts impacted 
eost, schedule, and system performanee. 

The United States General Accounting Office (GAO) released a report to 
congress in November of 1969 stating improved management of the Sheridan weapon 
systems, M60A1E1 tank turrets, and M60A1E2 tanks would have minimized the 
situation (GAO, 1969, November). 

The status on the aequisition of major weapon systems released by GAO in 
Eebruary, 1970 stated causes for the eost growth included (1) beginning acquisition 
before the weapon system had been demonstrated with probable successful development, 
(2) inadequate initial requirements and performanee specifications, and (3) the difficulty 
to anticipate cost growth on major weapon systems that are produced over a long period 
of time (GAO, 1970). 

Cost growth for the 61 systems in the comprehensive report in 1971 was 
attributed to various reasons: difference between planning and development estimates, 
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changes in quantities, engineering ehanges, revisions to estimates, and inflation (GAO, 
1971, Mareh). Even more speeifieally, there were eertain fundamental problems: (a) 
identifying the need for a weapon system and prioritizing, (b) determining performanee 
eharaeteristies and their feasibility, (e) quality suffering in support of eost-effeetiveness, 
(d) organizational strueture, and (e) eost estimation. 

The GAO (1971, Mareh) reeommended four distinet ehanges for improvement: 
(1) have a standardized proeedure for all military branehes to request new systems based 
on mission need and to prioritize those systems, (2) institute standards for preparing, 
updating, and utilizing eost-effeetiveness studies, (3) give more deeision-making 
authority and eontrol of operations to the eorresponding military braneh, and (4) 
standardize aequisition reports to eontain summary of aoeeptability of the weapon, 
relation to and reeognition of other eomplementary weapons, and eurrent status. 

In Perry et al.’s (1971) findings, the data indieated that meeting performanee 
requirements was the main objeetive for programs. Sehedule had a lesser priority than 
performanee, and eost growth was allowable to faeilitate meeting performanee and 
sehedule goals (Perry et ah, 1971). Perry et al. (1971) isolated the eauses for eost growth 
in three broad eategories: teehnieal uneertainty, seope ehange, and eost estimating error. 
It was obvious those programs with longer durations and higher teehnologieal advanees 
were the ehief offenders of eost growth. However, aeross the board, teehnieal uneertainty 
affeeted all three faetors of eost, sehedule, and performanee. Nonetheless, eost growth on 
fifty pereent of the programs was attributed to seope ehange. 

By 1972, it was evident a leading souree of eost growth on major weapons 
systems was teehnieal uneertainty. In response to this, GAO eondueted a thorough 
examination of the polieies and praetiees of testing and released a report on the 
importanee of testing and evaluation in major weapon system programs (GAO, 1972). 
GAO examined 13 major weapons systems and eoneluded that testing objeetives were 
adequate but were offset by inadequate testing plans. Testing and evaluation were not 
eompleted in time, and although the generated reports were adequate, the value of these 
reports was inadequate due to the laek of planning and aetual testing. GAO 
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recommended testing be conducted before major decisions or milestones, gaining control 
of waivers from required testing, and creating concise summary reports for all levels of 
management (GAO, 1972). 


In many of the periodic updates to congress on selected major weapon systems, 
GAO repeatedly inserted the following statement: 

Overly ambitious performance requirements, combined with low initial 
cost predictions and optimistic risk estimates, lead almost inevitably to 
schedule slippages, performance degradations, and cost increases. 
Attempt to keep total program costs from rising lead to reductions in 
planned quantities that, in turn, increase unit cost. (GAO, 1975, p.24) 

The primary reason for a net decrease in the weapon systems chosen in 1975 was 
due to economic factors, primarily inflation (GAO, 1976). However, cost growth 
continued and in his statement to congress, Stolarowe proclaimed, “Cost growth of 
weapon systems is a highly complex and multi-faceted problem involving economics, 
military judgment and politics” (GAO, 1979, June, p. 4). Stolarowe stated the factors 
leading to increased costs were interrelated and impossible to separate. One factor was 
preliminary cost estimates. Stolarowe acknowledged over-optimistic cost estimates 
become more accurate and realistic the further along in the development cycle because 
more is known about the system. Technology was the second factor because high 
technology systems were more complex needing more research, development, design, 
testing, maintenance, support, etc. The third factor was inflation, and the fourth, funding, 
both causing production delays, changes in production rates, and production 
inefficiencies (GAO, 1979, June). 

GAO’s reports to congress in 1982 and 1983 had many broad and specific 
recommendations concerning major weapon system acquisitions (GAO, 1982; GAO, 
1983). First, risk must be minimized and system effectiveness ensured through improved 
testing and limitations on procurement funds until those risks and uncertainties are 
resolved. Second, congress could make better decisions if it had more information on 
cost, schedule, and performance of major weapon systems provided by the Department of 
Defense. Third, requirements should be established in accordance with mission needs 
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and maintained through completion of the program. Fourth, more analyses of alternatives 
must be conducted to ensure the weapon system is the best all around. Fifth, every 
opportunity must be taken to ensure costs are reduced. Finally, a more effective 
management structure must be established to efficiently accomplish program objectives 
(GAO, 1982; GAO, 1983). 

From research conducted by the General Accounting Office, Charles A. Bowsher 
(GAO, 1984) reasoned cost growth could be more controlled if cost estimating 
procedures and reporting of actual costs were improved. Bowsher recognized the 
vagueness and conflicting guidelines for cost estimating. He conveyed numerous reasons 
for cost growth to the Senate: DoD’s implementation of the guidance was weak, cost 
estimates lack definite structure, definition of programs lacks consistency, documentation 
was insufficient, risks were not properly accounted for, inflation was not consistently 
recognized ,or it was used to hide other costs, improvements were needed in the direction 
provided by DoD, DoD estimates were based on inaccurate or optimistic data, program 
cost estimates were forced to conform to the budget, relevant program costs were 
excluded, and cost estimates would be more reliable by using independent cost estimator 
recommendations (GAO, 1984). 

The 1988 GAO report summarized 18 years of systemic causes of cost growth, 
schedule delays, and performance shortfalls. Several factors were highlighted: (1) no 
funding for analyzing alternatives, which forces one design; (2) premature commitment 
to an unproven design; (3) decisions made during program planning limit program 
managers; (4) lack of upper-management commitment to weapons requirements for 
design; (5) insufficient funding; (6) inappropriate external management direction; (7) 
more commitment to the development schedule existed than commitment to correcting 
serious problems; (8) process used by DoD to determine weapon system requirements; 
and (9) general instability with the acquisition process (GAO, 1988). 

In 1992, GAO attributed many of the persistent, systemic problems to “a 
prevailing culture that is dependent on generating and supporting new weapons 
acquisitions” (GAO, 1992, p. 9). This culture created an environment encouraging the 

selling of programs, and GAO suggested changes be made to the system of incentives 
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and opportunities surrounding the aequisition proeess (GAO, 1992). This was reiterated 
in GAO’s 1997 report where GAO reviewed six weapons in low-rate produetion (TRIP) 
and 22 weapons in full-rate produetion (GAO, 1997). DoD tended to buy large quantities 
of untested weapons in TRIP, whieh eaused many deereases in produetion rates of tested 
and proven weapons in full produetion. These proven weapons not only experieneed 
redueed produetion rates but also extended sehedules, whieh, in turn, resulted in higher 
eosts per unit. The larger investment in untested weapons resulted in inereased eosts and 
performanee risks. They suggested DoD buy limited quantities of weapons during TRIP 
until appropriate operational tests and evaluations were eondueted (GAO, 1997). 

GAO (1998) eompared eommereial praetiees to DoD praetiees eoneeming 
teehnology development. Unlike leading eommereial firms, DoD eonsistently began or 
eontinued programs without the needed teehnology fully developed, as mentioned in the 
numerous, previous reports. GAO observed DoD aequisition had a eulture or 
environment that eneouraged sueh praetiees, whieh led to high eosts and sehedule 
inereases. Reeommendations ineluded (1) having standards applied to individual 
programs eoneeming timing and quality of produetion-related knowledge, (2) splitting 
foreed a line of text to next page for more text at top of page teehnology development 
from weapon systems programs for higher knowledge standards, and (3) allowing 
program managers to identify unknowns as high risks early without eritieism or loss of 
funding (GAO, 1998). 

In her statement to the Subeommittee on AirLand, Committee on Armed Serviees, 
U.S. Senate, Sehinasi ealled for immediate ehange to the aequisition system, pleading 
with deeision makers to make the neeessary diffieult ehoiees and deeisions based in 
reality and to stop being eomplaeent to eost growth, sehedule slippages, and performanee 
shortfalls (GAO, 2005). Sehinasi aeknowledged DoD’s poliey had adopted a knowledge- 
based, evolutionary approaeh. However, DoD’s praetiee was the opposite. Sehinasi 
reeommended tmly implementing the knowledge-based, evolutionary approaeh to 
aequisition to enable “developers to rely more on available resourees rather than making 
promises about unproven teehnologies” (GAO, 2005, p. 6). This poliey fostered a more 
manageable environment, sinee teehnology would not be brought into produet 
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development unless it was first demonstrated to meet requirements. Nonetheless, 
programs eontinued to start development with immature teehnologies. 

To emphasize the need for deeision makers to begin making the neeessary but 
diffieult deeisions, Sehinasi (GAO, 2005) outlined the eurrent, undiseiplined progression 
to produet development: 

. DoD’s requirements ereate more demand for programs than resourees. 

. DoD approves too many highly eomplex and interdependent programs. 

. After a program is approved, DoD adds more requirements, inereasing 
eosts and risks. 

. Onee programs start, the budgeting proeess exaeerbates the problems. 

. Programs are funded annually and department wide. 

. Priorities have not been established. 

• Competition for funding inereases. 

• Sueeess is viewed by obtaining next ineremental funding, not by 
delivering eapabilities. 

• Bad news about programs is suppressed and testing ean be skipped to 
lower eosts, so as not to lose funding. 

• Senior offieials make aeross-the-board euts rather than make hard 
deeisions about individual programs. 

. Therefore, the aequisition system fosters a eulture of beginning programs 
with too many unknowns, risks, and, eventually, shortfalls and higher 
eosts (GAO, 2005). 

Sehinasi (GAO, 2005) gave speeifie reeommendations for improvement: (a) 
eonstrain requirements; (b) proved definitive business eases for investments; (e) enable 
seienee and teehnology organizations to take the responsibility for teehnology; (d) 
develop a workforee to manage requirements, souree seleetion, and knowledge-based 
aequisitions; and, finally, (e) enforee knowledge eapture and use by deeision makers 
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before investing or moving forward with programs. These reeommendations demanded a 
diseiplined proeess for requirements and funding (GAO, 2005). 

The 2006 report issued by GAO found DoD still skipping important steps in the 
knowledge-based proeess and eontinuing to ehase “revolutionary leaps in eapability” 
before gathering pertinent knowledge and before teehnology had matured enough. In 
faet, 80% of the programs reviewed for this report had not followed the knowledge-based 
proeess before eommitting to system development (GAO, 2006). 

D. INITIATIVES TO IMPROVE ACQUISITION OF MAJOR WEAPON 
SYSTEMS 

The perennial proeurement and eontraeting eyele—going baek many 
deeades—of adding layer upon layer of eost and eomplexity onto fewer 
and fewer platforms that take longer and longer to build must eome to an 
end. There is broad agreement on the need for aequisition and eontraeting 
reform in the Department of Defense. There have been enough studies. 

Enough hand-wringing. Enough rhetorie. Now is the time for aetion. 

United States Seeretary of Defense, Robert Gates, April 6, 2009 

The hundreds of reports released by GAO, the RAND Corporation, and other 
researehers did not go unnotieed. Over the past 50 years, the United States has 
implemented numerous initiatives for improvement of the aequisition system. Over the 
past deeades, the fundamental nature of major weapon systems aequisition has ehanged 
signifieantly. Unfortunately, as proven in the literature above, even with perpetual 
reports and eommittees or initiatives, programs eontinued the eyele of underperforming, 
extending the sehedule, and inereasing the eosts. 

Needed improvements were reeognized as early as 1949 with the formation of the 
first Hoover Commission. The first Hoover Commission, also known as the Commission 
on the Organization of the Exeeutive Braneh of Government, did not exaetly examine the 
aequisition system as a whole in 1949. However, it did review the role of the Seeretary 
of Defense within the proeess and the strueture of the military (MeKinney et ah, 1994). 
While the seeond Hoover Commission in 1955 was more eoneerned with eommodity 
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goods than weapon systems, it did review the aequisition system more thoroughly 
looking to aehieve greater effieieney within the proeurement system (MeKinney et ah, 
1994). 

During the 1960s, the U.S. Seeretary of Defense, Robert MeNamara, approaehed 
the U. S. aequisition system mueh as he did as president of the Ford Motor Company, a 
large Ameriean business (MeKinney et ah, 1994). MeNamara had a baehelor’s degree in 
eeonomies and philosophy from the University of California, Berkeley, and a master’s 
degree from Harvard Graduate Sehool of Business. MeNamara aggressively questioned, 
sought alternatives, and stimulated progress. He was an advoeate of “one defense 
poliey,” not three eonflieting polieies for eaeh of the military branehes (DefenseLink, 
2009). Some of MeNamara’s innovations, sueh as the Total Paekage Proeurement, had 
unintended eonsequenees like budget overruns. However, some programs sueh as the 
introduetion of a eampaign to reduee unneeessary duplieation within defense 
proeurement had positive impaets (MeKinney et ah, 1994). 

In 1970, the first independent, government sponsored eommission to thoroughly 
examine the weapon systems aequisition proeess was the Fitzhugh Commission, also 
known as the Blue Ribbon Defense Panel. The Fitzhugh Commission was also the first 
to address the politieal aspeets of the aequisition system by aeknowledging Congress’ 
tendeney to mieromanage the Department of Defense. The Fitzhugh Commission 
reeommended management improvements and ehanges in eontraeting regulations 
(MeKinney et ah, 1994). A year later in 1971, the introduetion of DoDD 5000.1 was a 
major breakthrough in the aequisition proeess. It illustrated three eritieal transition 
points, whieh were later referred to as milestones; (1) Program Initiation, (2) Begin Full- 
Seale Development, and (3) Begin Produetion/Deployment (Aequisition History Projeet, 
2009). 

Two years after the report issued by the Fitzhugh Commission, 1972, the 
Government Proeurement Commission also reeommended management improvements 
and ehanges in eontraeting regulations. However, what set the Government Proeurement 
Commission apart from the Fitzhugh Commission was the reeommendation that 
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Congress have a better understanding of defense as a whole and a more strategie role in 
defense aequisition (MeKinney et ah, 1994). 

The Carlueei Initiatives of 1981 had an aggressive agenda; reduee length of 
aequisition proeess, inerease eost savings, improve readiness, and strengthen the defense 
industrial base (Aequisition History Projeet, retrieved 2009). After reviewing the entire 
system, 32 initiatives were implemented. See Table 2 for a summary of the Carlueei 
Initiatives. 

Table 2. Carlueei Initiatives 


B 

Initiative 

■ 

Initiative 

1 

Reaffirm Acquisition Management Principles 

17 

Decrease DSARC briefing and data 
requirements 

2 

Increase use of Preplanned Product Improvement 

18 

Budget for inflation 

3 

Implement mutliyear procurement 

19 

Forecast business base conditions 

4 

Increase program stability 

20 

Improve source selection process 

5 

Encourage capital investment to enhance 
productivity 

21 

Develop and use standard operation and 
support systems 

6 

Budget to most likely costs 

22 

Provide more appropriate design-to-cost- 
goals 

7 

Use economicial production rates 

23 

Implement acquisition process decisions 

8 

Assure appropriate contract type 

24 

Reduce number of DSARC milestones 

9 

Improve system support and readiness 

25 

Submit MENS with Service POM 

10 

Reduce administrative costs and time 

26 

Revise DSARC membership 

11 

Budget for technological risk 

27 

Retain USDR&E as Defense Acquisition 
Executive 

12 

Provide front-end funding for test hardware 

28 

Raise dollar threshold for DSARC review 

13 

Reduce governmental legislation related to 
acquisition 

29 

Integrate DSARC and PPBS process 

14 

Reduce number of DoD Directives 

30 

Increase PM visibility of support resources 

15 

Enhance funding flexibility 

31 

Improve reliability and support 

16 

Provide contractor incentives to improve 
reliability 

32 

Increase use of competition 


[From Holbrook, 2003, p. 10]. 
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The Reagan administration ereated the Graee Commission, also known as the 
President’s Private Seetor Survey on Cost Control, to examine the entire government, 
ineluding defense aequisition, looking for ways to avoid wasteful publie spending. In 
their report of 1983, the Graee Commission eritieized the “exeessively eomplex set of 
regulations” surrounding the aequisition proeess and, mueh like the Fitzhugh 
Commission, eritieized Congress for mieromanaging weapons aequisitions that have a 
eost impaet for the interferenee. Later the Paekard Commission built on the Graee 
Commission’s eall for greater stability in defense programs and budget (MeKinney et ah, 
1994). 

Former President Ronald Reagan, as well, ehartered the Paekard Commission, a 
blue ribbon eommission (Aequisition History Projeet, retrieved 2009). The Commission 
reviewed the management of the aequisition proeess, the budget proeess, and the 
legislative oversight. After studying the defense aequisition system, the Paekard 
Commission reeommended the appointment of a full-time Under Seeretary of Defense 
for Aequisition, whieh was put into poliey in 1986 (Aequisition History Projeet, retrieved 
2009). Furthermore, the Paekard Commission ealled for unity and trust among Congress, 
the Exeeutive Braneh, the military, and defense industries (MeKinney et ah, 1994). 

Like many other initiatives and reports before it, the Goldwater-Niehols Aet of 
1986 mandated revolutionary reform for national defense. It was the Goldwater-Niehols 
Aet that aetually implemented most of the reeommendations by the Paekard Commission 
(Jones, 1999). 

At the time of this writing, the latest reform effort was passed into law on May 22, 
2009. The Weapon Systems Aequisition Reform Aet of 2009 was a bill proposed to 
improve the organization and proeedures of DoD eoneerning the major weapon systems 
aequisition. This new law ereated a position within DoD—Direetor of Cost Assessment 
and Program Evaluation. This new position will provide adviee relating to aequisition 
program eost estimation and analysis to the Seeretary of Defense and DoD offieials 
(Library of Congress, 2009). 
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“Poor execution of the revised acquisition policy is a major cause of DoD’s 
continued problems” (GAO, 2006). See Table 3 for a partial listing of programs executed 
for the purpose of improving acquisition system. 


Table 3. Acquisition Improvement Initiatives 


Year 

Improvement Initiatives 

1949 

First Hoover Commission 

1953 

Rockefeiier Committee 

1955 

Second Hoover Commission 

1961 

McNamara Initiative 

1970 

Fitzhugh Commission / Biue Ribbon Defense Panei 

1971 

DoDD 5000.1 was issued 

1972 

Commission on Government Procurement 

1978 

Defense Science Board Acquisition Cycie Study 

1979 

Defense Resources Management Study 

1981 

Cariucci Initiatives 

1983 

Grace Commission / President's Private Sector Survey on Cost Controis 

1986 

Packard Commission / Presdident's Biue Ribbon Defense Commission 

1986 

Goidwater-Nichois Act 

1989 

Defense Management Review 

1994 

Process Action Team on Oversight and Review 

2009 

Weapon Systems Acquisition Reform Act 


E. TOP MILITARY SPENDING COUNTRIES 

The United States is the largest consumer of military goods in the world. 
According to the Stockholm International Peace Research Institute (SIPRI) (2008)—a 
Swedish-based independent research institute focusing on international security, arms 
control, and disarmament—the United States’ military acquisition expenditures comprise 
41.5% of the world’s total military expenses. The top five military spending countries, 
which include the U.S.’s 41.5%, make up 60% of the world market share. See Table 4 for 
details on the top 15 military spending countries as of 2008. 
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Table 4. The Top 15 Military Spender Countries in 2008 


Rank 

Country 

Spending 

($b.) 

World 
Share (%) 

Spending 
per Capita 
($) 

Share of 

GDP, 2007 


1 

USA 

607 

41.5 

1967 

4 

66.5 

2 

China 

[84.9] 

[ 5 . 8 ] 

[63] 

[ 2 . 0 ] 

194 

3 

France 

65.7 

4.5 


2.3 

3.5 

4 

UK 

65.3 

4.5 


2.4 

20.7 

5 

Russia 

[58.6] 

[ 4.01 

[413] 

[ 3 . 5 ] 

173 

Sub-total top 5 

882 

60 




6 

Germany 

46.8 

3.2 

568 

1.3 

- 11.0 

7 

Japan 

46.3 

3.2 

361 

0.9 

- 1.7 

8 

Italy 

40.6 

2.8 

689 

1.8 

0.4 

9 

Saudi Arabia'’ 

38.2 

2.6 

1511 

9.3 

81.5 

10 

India 

30.0 

2.1 

25 

2.5 

44.1 

Sub-total top 10 

1 084 

74 




11 

South Korea 

24.2 

1.7 

501 

2.7 

51.5 


Brazil 

23.3 

1.6 

120 

1.5 

29.9 

■9 

Canada 

19.3 

1.3 

581 

1.2 

3.4 


Spain 

19.2 

1.3 

430 

1.2 

37.7 

■B 

Australia 

18.4 

1.3 

876 

1.9 

38.6 

Sub-total top 15 

1 188 

81 




World 


1 464 

100 

217 

2.4 

44.7 


[ ] = estimated figure; GDP = gross domestic product. 

“ The figures for national military expenditure as a share of GDP are for 2007, the most recent year for which 
GDP daa is available. 

“ The figures for Saudi Arabia include expenditure for public order and safety and might be slight 
overestimates. 


Due to public access and/or language barriers, it was not possible to gather 
information on all of the above countries’ defense acquisition systems. However, basic 
information about defense acquisition was found on seven of the top 15 countries: 
United States, France, Germany, United Kingdom, Australia, Japan, and South Korea. 

1. The United States’ Defense Acquisition System 

According to the CIA world Factbook, the United States (U.S.) has the largest 
economy in the world. The U. S. has a constitution-based federal republic government 

with a strong democratic tradition. It has a population of 307,212,123. The U.S. Armed 
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Forces has four branches; U.S. Army, U.S. Navy including the Marine Corps., U.S. Air 
Force, and U.S. Coast Guard (CIA World Factbook, 2009). 


The U.S. political system has a series of checks and balances concerning DoD 
weapon systems acquisition. Congress plays a significant role in oversight and controls 
the “power of the purse” (Kausal, 1999, p.4-10). The DoD requests specific weapon 
systems and justifies the need. From this, the President submits a budget to congress. 
After deliberation and modifications, it may be passed, and then the budget is sent back 
to the President for his signature or veto. This process typically takes eight months. See 
Figure 1 for the typical flow of budget for the U.S (Kausal, 1999). 



Figure 1. Typical Flow of Budget [From; Kausal, 1999]. 


The DoD has three decision-making support systems for defense acquisition; Joint 
Capabilities Integration and Development System (JCIDS); Planning, Programming, 
Budgeting, and Execution (PPBE) System; and Acquisition Management System (DoD 
5000.02). Eigure 2 depicts these three systems enclosed within the “Big ‘A’ Acquisition 
Process.” 
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LITTLE “a ■ 
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(MOST PREVIOUS EFFORTS 
FOCUSED HERE) 


Figure 2. The Aequisition System [From; Defense Aequisition Performanee Assessment 


Report, 2006, p. 4]. 


The Joint Capabilities Integration and Development System determines future 
military needs. Eaeh of the military serviees is responsible for identifying gaps within its 
own mission requirements, determining new mission needs, and generating requirements 
to meet those needs (DAU, 2009). All major weapon systems aequisitions, also known 
as Aequisition Category (ACAT) I programs, are founded on validated mission needs 
where the requirement is approved by the Joint Chiefs of Staff. The aequisition system 
is managed by the Under Seeretary of Defense (Aequisition and Teehnology) (Kausal, 
1999). 

The Planning, Programming, Budgeting, and Exeeution System helps with 
ereating DoD’s pieee of the President’s Budget (DAU, 2009). Three of the phases— 
planning, programming, and budgeting—^usually take three years. Planning is a six- 
month proeess and begins two years before the fiseal year it will be requested. During 
Programming, the military branehes issue their Program Objeetive Memoranda (POM), 
stating their requirements to the Joint Chiefs of Staff (JCS). The military branehes, the 
JCS, and the Offiee of Seeretary of Defense diseuss over a period of time and agree to a 
number of programs. The Seeretary of Defense then issues a Program Deeision 
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Memorandum (PDM) (Kausal, 1999). Finally, in the Budgeting phase, the DoD budget 
is finalized and submitted to be ineluded in the President’s budget (Kausal, 1999). 

The Acquisition Management System (DoD 5000.02) contains the policies and 
procedures guiding DoD and consists of five phases, three milestones, and three decision 
points. Once the Materiel Development Decision has been made, the Milestone Decision 
Authority will allow entrance into any point in the system as seen in Figure 3 as long as 
phase-specific entrance criteria and statutory requirements have been met (DoDl 5000.02, 
2008). 
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Figure 3. Defense Management Acquisition System [From; DoDl 5000.02, 2008]. 


The United States has more than 80 major weapon systems currently in 
development (DoD FY2009 Budget Request Summary Justification). The acquisition of 
major weapon systems represents 20 percent of the Department of Defense’s annual 
spending on purchases (Hearing on the Reform of Major Weapon Systems Acquisition, 
Defense & Security News—ByU.S.Government on May 5, 2009). 

2, France’s Defense Acquisition System 

According to the CIA World Factbook (2009), France has the eighth largest 

economy in the world. France’s government is a republic, and the country has a 

population of 64,057,792. France has four military branches; Army, Navy, Air Force, 

and National Gendarmerie (CIA World Factbook, 2009). 
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Parliament creates the laws concerning the defense budget, military policy, and 
defense organizational structure (Kausal, 1999). The Ministry of Defense implements 
these policies with the aid of the Joint Armed Forces Chief of Staff, the Delegation 
General for Armaments (DGA), and the Secretariat General for Administration (Kausal, 
1999). See Figure 4 for more detail. 



Figure 4. Organization of National Defense [From; Kausal, 1999]. 


The DGA was created in 1961 and is responsible for France’s weapon systems 
acquisitions (Kausal, 1999). Like the U.S., France has had problems of schedule delays 
and cost overruns with its complex, major weapons acquisitions. Also like the U.S., 
France has undergone acquisition reform to address those issues and to allow for greater 
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flexibility in the process. The new acquisition process consists of stages and phases. The 
four stages are preparation, design, realization, and utilization (Kausal, 1999) (see Figure 
5). 



Figures. New Acquisition Process. 


During the Preparation Stage, operational requirements are outlined while current 
assets are evaluated. Recommendations are given to fill the identified gaps such as 
developing advanced technology or buying off-the-shelf solutions. Throughout this 
preliminary stage, many technical and financial studies are made, and research and 
development programs are initiated. By the end, operational requirements have been 
refined and outlined, resources have been decided, and preliminary budgets have been set 
(Kausal, 1999). 

At the Design Stage, a decision is made, so the Feasibility and Definition Phases 
can begin. A program director and a program officer are appointed when a joint decision 
has been made to move forward. At this point, 80% of the costs of the program will be 
determined, leaving 20% for disposal during the development stage. The defense 
industry submits proposals. Once operational and technical specifications are defined, 
the defense industry will compete for the contract (in most cases), and an industry partner 
is chosen (Kausal, 1999). 
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The Realization (Development/Industrialization and Produetion) Stage involves a 
formal sehedule, performanee requirements, and a set budget. A very rigorous proeedure 
is put into plaee for the development and development tests, ensuring no duplieation 
between the eontraetor and DGA. The government firmly eommits to a multi-year 
eontraet with the eontraetors. This gives the eontraetors stability needed to invest in 
produetion at lower eost (Kausal, 1999). 

Finally, the Utilization Stage begins after a weapon system has been formally put 
into operational use. This, of eourse, is preeeded by mueh testing and aeeeptanee by the 
DGA. During the eourse of the system’s lifetime, it is assessed for needed updates, 
ultimately resulting in engineering ehanges (Kausal, 1999). 

3. Germany’s Defense Acquisition System 

According to the CIA World Factbook, Germany has the fifth largest economy in 
the world (in purchasing power parity), only behind the U.S., China, Japan, and India. 
Germany is a federal republic with a population of 82,329,758. Germany’s military, the 
Federal Armed Forces, has five separate branches; Army, Navy, Air Force, Joint Support 
Services, and Central Medical Service (CIA World Factbook, 2009). 

Germany has a Federal President, who is Head of State, as well as a Chancellor, 
who is Head of the German Federal Government (Kausal, 1999). The Chancellor plays 
two roles in defense; (1) sets policy for military issues and (2) in times of war, becomes 
the power of command. Each of the individual military branches develops requirements 
and submits them to the Director General of Armaments (DGA) for review and approval. 
The DGA is also responsible for providing targets for cost, schedule, and performance. 
The Federal Office of Military Technology, which falls under the Federal Ministry of 
Defense (FMOD), is a civilian-run operation and is in charge of weapon systems 
acquisitions, among other responsibilities. However, all budgets for major weapon 
systems acquisition must be approved by Parliament (Kausal, 1999). 

Major weapon systems are developed in phases; Pre-Phase, Definition Phase, 
Development Phase, Procurement Phase, and In-Service Phase. See Figure 6 for more 
detail. 
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Figure 6. Weapon Systems Development Proeess [From: Kausal, 1999, p. 2-21], 


At the end of each phase and before entering a new phase, a decision and approval 
are required before the program can continue (Kausal, 1999). The Pre-Phase verifies the 
military mission needs. During this phase, military and industry work together to 
evaluate needs, solutions, and alternatives. Once a solution has been finalized, submitted 
for review, and approved. Definition Phase begins. Definition Phase finalizes weapon 
specifications and establishes the project manager and team working groups. It is only 
finished when the Development Baseline is approved. After this approval, the 
Development Phase begins. At some point in this phase, the prime contractor is chosen 
through a fair and open competition, with the development contract defining the 
responsibilities of the contractor. Operational testing is conducted during this phase, and 
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it concludes with the approval of the doeument, Approval for Produetion. The 
Proeurement Phase ineludes series produetion and the seleetion of a eontraetor for the 
entire proeurement phase, again this is aeeomplished through fair and open eompetition. 
At the end of the Proeurement Phase, the weapon system is handed over to the military, 
whieh begins the In-Serviee Phase (Kausal, 1999). 

4. The United Kingdom’s Defense Acquisition System 

The United Kingdom (U.K.) has the sixth largest eeonomy, aeeording to the CIA 
World Faetbook (2009). The U.K.’s government is a eonstitutional monarehy and 
Commonwealth realm. The population is 61,113,205. The U.K.’s military branehes are 
the Army, Royal Navy, and Royal Air Foree (CIA World Faetbook, 2009). 

The U.K. has both a Monareh and a Prime Minister. The Monareh is the Head of 
State and the Commander in Chief of the Armed Forees. However, she is eonstitutionally 
bound to take adviee from the Prime Minister. It is the House of Commons that formally 
approves funds for weapon systems aequisition. However, the House of Commons 
approves the budget only, it does not have a say on individual programs. Individual 
major weapon systems needs and requirements are determined by the Equipment 
Approvals Committee, who eompares eosts to operational effeetiveness, as well as other 
trade-off analyses. The eommittee then gives reeommendations to the Ministry of 
Defenee. Also, the Defenee Evaluation and Researeh Ageney eonduets researeh into 
advaneed teehnology and makes reeommendations to the Ministry of Defenee (Kausal, 
1999). 

The aetual aequisition and management of major weapon systems are the 
responsibilities of the Defenee Proeurement Ageney, whieh is aeeountable to Parliament. 
It is the Ageney’s goal to obtain the best value for money and this is aeeomplished 
through eompetitive bids. Eoreign eontraetors are free to bid on the majority of defense 
aequisitions, as the prime eontraetor or as subeontraetors. Eike the U.S., the U.K. has had 
issues in eost, sehedule, and performanee on major weapon systems aequisition. 
Therefore, after mueh review, the Eabour Government ereated the Smart Proeurement 
Initiative in 1998, whieh takes a systems approaeh to aequisitions, improves requirements 

29 



management, strengthens partnerships with industry, controls schedule, commits to 
longer periods for acquisition funding, and implements incremental acquisition (Kausal, 
1999). 

The U.K.’s new acquisition system reduces the number of formal approvals and 
phases. The acquisition system begins with the Concept Phase to identify best 
alternatives for mission needs. Approval for the necessary resources is given at the Initial 
Gate before beginning the Assessment Phase. During the Assessment Phase, a single 
option is chosen from alternatives for demonstration. Also during this phase, technical 
risks are reduced to an acceptable level, requirements are set and linked to mission needs, 
firm life-cycle costs are set, and up to 15% of total project costs can be spent (Kausal, 
1999). 

The Main Gate is the major approval point where commitment to an individual 
project is made. It is at this point that projects are cancelled, if they do not provide an 
acceptable trade-off between cost and performance—best value for money. If approval is 
given, then the program proceeds to the Demonstration Phase where a contractor is 
selected and a contract is signed for development and production. Right through the 
Demonstration Phase, design continues as requirements and performance are met at a 
fixed cost. Models or prototypes are built to demonstrate integration and capabilities. 
The Manufacture Phase starts production of the system. The system will be tested by the 
manufacturer and end-user before it is officially accepted and put into service. See 
Figure 7 for more detail. 



Figure 7. The U.K.’s Defence Procurement Phases [From; Kausal, 1999, p. 3-24]. 
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5. 


Australia’s Defense Acquisition System 


The island, country, and continent—^Australia—has the 18* largest economy 
compared to other countries (CIA World Factbook, 2009). Australia’s government is a 
federal parliamentary democracy and a Commonwealth realm. It has a population of 
21,262,641. The Australian Defense Force has four braches: Australian Army, Royal 
Australian Navy, Royal Australian Air Force, and Special Operations Command (CIA 
World Factbook, 2009). 

Australia has the world’s largest coastline to defend, yet it has no identifiable 
threat. Therefore, Australia struggles with how to structure defense as well as how much 
to invest. Australia’s Defence Organization is the combination of the Australian Defence 
Force (ADF) and the Department of Defence. The Chief of Defence Force (CDF) is in 
command of the ADF, and the Secretary of Defence is head of the Department of 
Defence. Both of these positions are accountable to the Minister of Defence and the 
government for the management of Defence. Like many countries, Australia’s defense 
acquisition system has undergone major reform. To gain better understanding and 
control of acquisition, a new structure was created for the Defence Acquisition 
Organisation (DAO) (Kausal, 2000). See Figure 8 for more detail. 
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Figures. Defense Acquisition Organizational Structure [From; Kausal, 2000, p. 1^1]. 


Australia’s weapon systems acquisition process has three main phases; Pre¬ 
project Approval Phase, Approval Phase, and Post-Project Approval Phase. The Pre- 
Project Approval Phase deals with capability development, mission needs, program 
definition, analysis of alternatives, acquisition strategy, military priorities, and approval 
for proposed programs. The Post-Project Approval Phase, also known as the 
Implementation Phase, asks for bids from industry, evaluates and selects contractors. 
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manages the acquisition and contract, and oversees the weapon systems’ acceptance into 
operational service. See Figure 9 for more detail on Australia’s acquisition life-cycle. 
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Figure 9. Australia’s Acquisition Life-cycle [From; Kausal, 2000, p. 1^9]. 


Australia’s once publically owned defense industry has been privatized as a 
means to create more efficiency. Foreign competitors are welcomed to bid. Flowever, 
Australia stipulates certain long-term commitments and investments from these firms. 
The defense culture has changed tremendously in Australia. Now, local industry is 
increasingly involved in capability development, capability management, and cost 
estimates (Kausal, 2000). 

6, Japan’s Defense Acquisition System 

Compared to other countries, Japan has the third largest economy with a 
population of 127,078, 679 (CIA World Factbook, 2009). Japan has a parliamentary 
government with a constitutional monarchy. The Japanese Ministry of Defense oversees 
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three military branehes; Ground Self-Defense Force, Maritime Self-Defense Force, and 
Air Self-Defense Force (CIA World Factbook, 2009). 

Although Japan’s Prime Minister is the head of government, the executive power 
of the state is held by the Cabinet. The Cabinet is comprised of the Prime Minister and 
other Ministers of State who are appointed and removed by the Prime Minister. Part of 
the Office of the Prime Minister is Japan’s Defense Agency. This is led by the Director 
General, who is the Minister of State for Defense (Kausal, 2000). See Figure 10 for more 
detail on the structure of Japan’s Defense Agency. 



Figure 10. Japan’s Defense Agency [From; Kausal, 2000, p. 2-15]. 


Weapon systems acquisition in Japan is primarily managed by three bureaus; the 
Bureau of Defense Policy, the Bureau of Finance, and the Bureau of Equipment. The 
Bureau of Defense Policy is responsible for designing defense policy and programs, 
conducting trade-off analyses, and defining the military’s operational activities. The 
Bureau of Finance is involved in creating the defense budget and military spending 
priorities. The Bureau of Equipment is responsible for oversight and management of the 
acquisition system (Kausal, 2000). 
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In Japan, mission needs are defined by the individual military branehes and 
proposed to the defense department in the form of new equipment needs or changes to 
existing systems. At this point, an Operational Requirements Document is prepared and 
the Research and Development Process begins, which consists of four phases and 
initializes the acquisition process; Concept Phase, Research Phase, Development Phase, 
and Operation Phase. The Concept Phase conducts feasibility studies, defines costs, and 
analyzes technical challenges. The Research Phase identifies technical risks, 
demonstrates and tests technology, and performs sub-system research. The Development 
Phase requires the contractor to design the system, build a prototype, and test for 
performance. If the system performs acceptably, then it is ready for the production 
phase. However, funding must first be secured and commitment gained from the Defense 
Agency and others. The Operation Phase begins with operational tests. Once these are 
successful, approval is given to award contracts for production (Kausal, 2000). See 
Figure 11 for more information on Japan’s acquisition system. 
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Figure 11. Japan’s Acquisition System [From; Kausal, 2000, p. 2-26]. 
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7. 


South Korea’s Defense Acquisition System 


South Korea has the 14th largest eeonomy in the world eompared to other 
eountries (CIA World Faetbook, 2009). It is a republic with a population of 48,508,972. 
South Korea’s military has three branches; Republic of Korea Army, Navy, and Air 
Force (CIA World Faetbook, 2009). The President is elected to a single, five-year term 
and is the Flead of State, Chief Executive, and Commander-in-Chief of the Armed 
Forces. The Ministry of National Defense is one of the President’s responsibilities. It 
was reorganized in May 1999 (Kausal, 2000). See Figure 12 for the organizational 
structure of the Ministry Of National Defense. 



Figure 12. The Republic of Korea’s Ministry of National Defense [From; Kausal, 2000, 

p. 3^]. 


The Ministry of National Defense has recently reformed its acquisition system, 
introducing a new acquisition development structure in February 2000. Korea’s 
acquisition structure consists of the Defense Acquisition Office, the Planning and 
Management Office, and the Policy Office, all of which fall under the Minister and Vice 
Minister of National Defense. This new development structure was formed to reduce 
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processes and time, faeilitate eeonomieal aequisition, and establish elear lines of 
communieation (Kausal, 2000). See Figure 13 for a more detailed view. 



Figure 13. Korea’s Aequisition Development Strueture [From; Kausal, 2000, p. 3-17] 


Requirements for new weapon systems can come from different sources, not just 
the military branches. However, much like the U.S., eaeh serviee does propose new 
weapon systems through the document. Required Operational Capability. Approval of 
major weapon systems must be obtained by the Joint Chiefs of Staff (JCS). The 
aequisition process consists of five phases; Planning, Programming, Budgeting, 
Exeeuting, and Evaluation. The Planning Phase first assesses the threat and then 
determines the strategy. Requirements are further explored and a Medium-Eong Range 
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Force Requirements Proposal is created. This proposal is evaluated by eertain members 
of the JCS; and if they agree on the proposal, they prepare the Joint Strategie Objeetive 
Plan. This is then reviewed by other members of the JCS; and if these members agree, 
they prepare the Mid-Term Defense Plan and the Defense Aequisition Development Plan. 
These two plans define the Programming Phase. During the Budgeting Phase, the 
Defense Budgeting Doeument is organized. In order to allow more stability within the 
programs, multi-year funding is requested for the entire projeet. Onee this is approved by 
the Minister of National Defense, it is submitted to the President’s Budget Administration 
Offlee in May. This is reviewed by the eabinet and then sent to the National Assembly 
for review and approval. Budget is usually passed in Deeember. The Execution Phase 
awards eontracts for research and development and for proeurement. The Evaluation 
Phase analyzes eompleted programs and provides feedback (Kausal, 2000). See Eigure 
14 for more detail on the Acquisition Process. 
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Eigure 14. Korean Ministry of National Defense Aequisition Process [Prom; Kausal, 

2000, p. 3-27]. 
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III. METHODOLOGY 


A. CLIOS PROCESS 

“When we try to piek out anything by itself, we find it hitehed to everything else 
in the universe.” -John Muir 

Complex systems with wide ranging eeonomie, soeial, and politieal impaets are 
more eommon than ever before, yet understanding their true nature and solutions for 
problems surrounding them is less known. A CLIOS system is one that is Complex, 
Large-seale, Intereonneeted, Open, and Soeio-teehnieal (Sussman, 2007). Professors and 
doetoral students in the Engineering Systems Division at MIT developed the CLIOS 
proeess for just sueh systems. It is now taught in the elass, “Frameworks and Models in 
Engineering Systems.” Engineering systems are ones with a teehnieal aspeet to them, not 
just eeonomie, politieal, or soeial. The CLIOS framework is an iterative proeess used to 
study CLIOS systems. It is a systems approaeh to analyzing and addressing problems by 
representing the entire system in an integrated form. 

Complexity is a key aspeet of a CLIOS system (Sussman, 2007). First, struetural 
eomplexity is an attribute of a system with many intereonneeted parts. Behavioral 
eomplexity is an attribute of a CLIOS system, e.g., it is diffieult to prediet system 
behavior. The internal behavior of subsystems and eomponents may be understood, yet 
the relationship among all of these subsystems and eomponents may be diffieult to 
prediet. Nested eomplexity oeeurs when a physieal or teehnieal system is embedded and 
internets with an institutional sphere. The institutional sphere is a system of its own, 
whieh may experienee struetural and behavioral eomplexity. It is this interaetion of the 
two systems—^physieal and institutional—^within one bigger system, whieh ereates nested 
eomplexity. Finally, evaluative eomplexity refers to the involvement of multiple 
stakeholders with multiple perspeetives, expeetations, and values (Sussman, 2007). 

In addition to being eomplex, CLIOS systems are large-seale, intereonneeted, 
open, and soeioteehnieal (Sussman, 2007). Large-seale means these systems have a large 
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impact, are long-lived, and usually are large-seale in geographie sense. A CLIOS system 
is intereonneeted meaning it is interrelated with other soeiotechnieal systems. Sussman 
(2007) gives the example of the relationship among transportation systems, energy 
systems, and the global climate systems as being intereonneeted. Above and beyond the 
teehnieal aspeet, a CLIOS system is also open, meaning it eonsists of soeial, politieal, and 
economic characteristics. Finally, a CLIOS system is soeiotechnieal in that it is not solely 
technical, nor is it exclusively soeial. See Figure 15 below. 



Figure 15. CLIOS System—Physieal Domain with Subsystems Embedded in an 

Institutional Sphere 


B. CLIOS STRUCTURE 

The basie strueture of the CLIOS analysis proeess is three stages eneompassing 
12 steps (Sussman, 2007) (See Figure 16 below). The three stages inelude (1) 
Representation; (2) Design, Evaluation, and Seleetion; and (3) Implementation. The 
twelve steps make up the process with Steps 1 through 5 falling within the Representation 
stage; Steps 6 through nine in the Design, Evaluation, and Seleetion stage; and Steps 10- 
12 in the Implementation stage: 
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1. Describe CLIOS System; Checklists & Preliminary Goal Identification 

2. Identify Subsystems in Physical Domain & Groups on Institutional Sphere 

3. Populate the Physical Domain & Institutional Sphere 

4. a.) Describe Components and b.) Describe Links 

5. Transition from Descriptive to Prescriptive Treatment of System 

6. Refine CLIOS System Goals & Identify Performance Measures 

7. Identify & Design Strategic Alternatives for System Improvements 

8. Identify Important Areas of Uncertainty 

9. Evaluate Strategic Alternatives & Select “Bundles” 

10. Physical Domain / Subsystems 

11. Institutional Sphere 

12. Evaluate, Monitor, & Adapt Strategic Alternatives for CEIOS System 
(Sussman, 2007) 
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Figure 16. CLIOS Process (From: Sussman, 2007, p. 15) 


The CLIOS Process requires analysts to take a strong systems-thinking approach. 
It provides a framework for understanding systems holistically and more clearly 
(Sussman, 2007). 
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1. Representation Stage 


The purpose of the representation stage is to explore and eome to an 
understanding of the strueture and behavior of the system being analyzed (Sussman, 
2007). This stage primarily utilizes diagrams to illustrate the system, subsystems, 
institutional sphere, and the interrelationships among them. At this point, there is a 
eommon understanding of the system, whieh leads to better understanding the issues and 
goals. 

The representation stage eneompasses the first five steps to illustrating and 
interpreting the system. Step 1 requires eheeklists and preliminary goal identifieation 
(Sussman, 2007). It is during this step that the CLIOS System is preliminarily bounded. 
The first eheeklist, the eharaeteristies eheeklist, may inelude the following; 

Temporal and geographie seale of the system, (b) the eore teehnologies 
and systems, (e) the natural physieal eonditions that affeet or are affeeted 
by the system, (d) the key eeonomie and market faetors, (e) important 
soeial or politieal faetors or eontroversies related to the system and (f) the 
historieal development and eontext of the CLIOS System. (Sussman, 

2007, p. 19) 

The seeond eheeklist requires deseribing opportunities, issues, and ehallenges 
(Sussman, 2007). These will be used in Stage 2 to determine improvements through 
strategie alternatives. The final eheeklist establishes preliminary system goals, whieh 
usually relate to the results from the seeond eheeklist (Sussman 2007). See Figure 17 
below. 


CliJiracteristics 

Checklist 


Opportimities Issues 
Challeuges Checklist 


Preliiniiian CLIOS 
System Goals 
Checklist 






















Figure 17. CLIOS System Cheeklists (From; Sussman, 2007, p. 20) 
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Since the CLIOS framework is an iterative proeess, system boundaries may 
expand or eontraet or even be redefined as the analysis progresses (Sussman, 2007). 

Step 2 of the representation stage requires the analyst identify subsystems in the 
physieal domain and groups in the institutional sphere (Sussman, 2007). There are three 
main aetivities in Step 2; (1) defining the major subsystems in the physieal domain, (2) 
determining the main aetor groups in the institutional sphere, and (3) defining the 
interrelationships between these on a maero-level. There are various ways to aeeomplish 
this. One is by using the information gathered in Step 1 and grouping the issues into 
eategories. Another is by elassifying the subsystems by eommon funetions, teehnologieal 
aspeets, or relationships to other aetor groups in the institutional sphere. For the physieal 
domain, the physieal system is broken out into subsystems, and the linkages among the 
systems are identified. This may be diffieult to develop but valuable due to the better 
understanding of the system strueture and behavior. For the institutional sphere, the 
information for the major aetor groups ean be obtained from the information gathered in 
Step 1 (Sussman, 2007). 

While Step 2 requires identifying subsystems in the physieal domain and aetor 
groups in the institutional sphere. Step 3 requires populating the physieal domain and 
institutional sphere (Sussman, 2007). For the physieal domain, early CLIOS diagrams 
are formed to detail eaeh of the subsystems with eomponents and links. There must be a 
balanee between adding too mueh detail and oversimplifying the subsystem and 
eomponents. The institutional sphere is also populated with individual aetors within 
major aetor groups, along with the links among them. 

Step 4A requires deseribing these eomponents and aetors in the physieal domain 
and institutional sphere respeetively that were earlier identified and diagramed (Sussman, 
2007). Within the physieal domain, the individual eomponents are defined and 
eategorized into one of three types of eomponents; regular eomponents, poliey levers, or 
eommon drivers. Regular eomponents are the most eommon and ean refer to eomplex 
struetures sueh as eeonomie growth or eoneepts sueh as eongestion. Poliey levers are 
eomponents that are infiueneed by deeisions made by the aetors in the institutional 

sphere. Common drivers are eomponents that are shared aeross subsystems. In addition, 
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there ean be external faetors, whieh may be a regular eomponent or eommon driver. To 
differentiate these in the diagram, CLIOS has three shapes and one shading. See Figure 
18. 




external 

factor 


Figure 18. Suggested CLIOS System Diagram Component Shapes (From; Sussman, 

2007, p. 24) 


The aetors in the institutional sphere are also deseribed as part of Step 4A. It is 
important to identify eharaeteristies sueh as their influenee over subsystems, their 
relevanee to the subsystems, their knowledge and resourees, their agenda eoneerning 
strategie alternatives, and other relevant information (Sussman, 2007). 

Step 4B requires the analyst to deseribe the links denoting direetionality of 
influenee, extent of influenee, timeframe of influenee, and uneertainty (Sussman, 2007). 
The CLIOS proeess has three elasses of links: (1) Class 1; links between eomponents, 
(2) Class 2: links between eomponents and aetors, and (3) Class 3; links between aetors. 
Figure 19 identifies suggested link shapes for CLIOS diagrams. 
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LINK 

SHAPE 

Class 1 (link between components of physical subsystems) 
Class 3 (link between actors on the institutional sphere) 


Class 2 (links ‘Trojecting” interactions between the 
institutional sphere and the physical domain) 


Weak 


Average 

» 

Strong 


Bi-directional 

M -► 

Positive (increase in component A results in increase in 
component B) 


Negative (increase in component A results in decrease in 
component B) 

-► 


Figure 19. Suggested Link Shapes (From; Sussman, 2007, p. 27). 


The final step in the representation stage is Step 5, whieh is a transition from 
deseriptive to preseriptive treatment of the system. At this point, the general strueture of 
the system has been determined, and the behavior of the eomponents, aetors, and links 
has been deseribed. Before proeeeding to Stages 2 and 3, the analyst should refleet and 
ask questions sueh as (1) are there strong interaetions within or between subsystems or 
aetors? (2) whieh eomponents are influeneed by many different aetors? and (3) are 
relationships between aetors full of eonfliet or eooperation? Many more questions should 
be asked to better understand the system as a whole before moving to the next stages 
(Sussman, 2007). 

2. Design, Evaluation, and Selection Stage 

Stage 2—Design, Evaluation, and Seleetion—moves from stage one where initial 
strueture and behavior were studied to the next stage where system eomplexity is 
examined in greater depth to find leverage points for improvement (Sussman, 2007). At 
this stage, strategie alternatives are developed. Where stage one was qualitative, this 
stage is more quantitative using models for eomparing strategie alternatives. 
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Stage 2 encompasses Steps 6 through 9 to design, evaluate, and select (Sussman, 
2007). During Step 6, the analyst refines the CLIOS system goals, which were initially 
created in stage one, and identifies performance measures. Performance measures 
identify progress made from the current state of the system to the desired future state as 
determined when refining the goals. Performance measures can be very difficult to 
define and prioritize. 

In Step 7, the analyst identifies and designs strategic alternatives for CLIOS 
system improvement (Sussman, 2007). Once goals and performance measures are 
established, the natural progression is to find areas of improvement. Identifying strategic 
alternatives requires creative thinking and brainstorming. Strategic alternatives are 
physical, policy-driven, or actor-based. A physical strategic alternative may be derived 
after bottom-up systems engineering approach is taken to analyze the physical domain, 
subsystems, and components. A policy-driven strategic alternative may be derived after 
analyzing how various policies influence the system. Finally, an actor-based strategic 
alternative may be derived after evaluating institutional arrangements. 

Step 8 requires flagging important areas of uncertainty (Sussman, 2007). This 
step may be conducted in parallel with Step 7 when identifying strategic alternatives for 
performance improvement. Information gained in Steps 1 through 6 may provide 
insights into uncertainties. Links with strong interactions may be an area of uncertainty. 
Conflict between actors may be another uncertainty. Common drivers may have strong 
influences and, therefore, be another area of uncertainty. Methods for determining 
uncertainty can be quantitative or qualitative such as risk assessments or scenario 
planning. 

The final step in stage two. Step 9, is to evaluate strategic alternatives and bundles 
(Sussman, 2007). Flere the strategic alternatives determined in Step 7 are evaluated using 
models developed in Step 6. Additional models may be necessary. Each of the strategic 
alternatives is evaluated by the impacts it has on the CLIOS system. Trade-off analysis 
may also be used to compare strategic alternatives, ft is rare for a single strategic 
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alternative to be ehosen and implemented due to the eomplexity of CLIOS systems. 
Usually, one alternative will not meet the CLIOS system goals. Therefore, strategie 
alternatives may be eombined into bundles. 

3. Implementation Stage 

The final CLIOS proeess stage is implementation (Sussman, 2007). Although 
bundles of strategie alternatives have been ehosen, implementation of those alternatives 
may have hurdles of its own. The implementation stage designs a guide for 
implementing the alternatives in Steps 10 through 11. Step 12 is the final step in the 
implementation stage and the CLIOS Proeess as a whole. 

Step 10 foeuses on designing and implementing the plan for the physieal domain 
and subsystems (Sussman, 2007). Many questions arise and must be answered during 
this phase before the plan is fully developed. Are there enough resourees? Is the time- 
frame for meeting system goals reasonable? How do the strategie alternatives fit with 
eaeh other? How do failures in meeting targets affeet the implementation? Whieh 
aotor(s) will implement, monitor, and enforee whieh strategie alternative? Attention must 
be given to the performanee measures and trade-offs. 

Step 11 foeuses on designing and implementing a plan for the institutional sphere 
and may be eondueted in parallel with Step 10 (Sussman, 2007). Some of the strategie 
alternatives developed in Step 9 may inelude aetor-based ehanges and must be ineluded 
in the overall implementation plan. The analyst must understand the aetor(s), the aetor’s 
goals, and the ehanges that will affeet the aetor(s). There may be opponents of these 
alternatives with resistanee to implementation. 

Step 12 ensures that the implemented alternatives are monitored and evaluated 
both in the short- and long-term (Sussman, 2007). Although the strategie alternatives are 
evaluated in Step 9, it is impossible to know their true impaets on the system as a whole 
until after implementation. It is possible to have unantieipated and/or indireet side 
effeets. During Stepl2, outeomes are observed, and neeessary modifieations are made. 
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c. 


CONCLUSION 


The CLIOS Process is iterative in nature. Goals and performance measures set in 
greater depth in Step 6 may require the analyst to return to Step 1 to review and challenge 
the preliminary goals. In Step 7 when identifying and designing strategic alternatives, the 
analyst may need to revisit Step 2 and refine or add additional subsystems and actors. 
Again, when flagging uncertainties in Step 8, Step 2 may be revisited and revised. 
Finally, in Step 12 where the strategic alternatives are monitored for impacts and 
outcomes, it may be necessary to return to Steps 5, 7, 8, and/or 9. See Figure 20 for the 
iterative process. 
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1. Describe CLIOS System: 
Checklists & Preliminary 
Goal Identification 



B 


2. Identify Subsystems in 
Physical Domain & Groups 
on Institutional Sphere 


3. Populate the Physical 
Domain & Institutional 
Sphere 



4A. Describe Components 


4B. Describe Links 


5. Transition from Descriptive to 
Prescriptive Treatment of System 


6. Refine CLIOS System 
Goals & Identify 
Performance Measures 


4 - 0 - 



7. Identify & Design Strategic 
Alternatives for System 
Improvements 

8. Identify Important Areas of 
Uncertainty 

*(D 


9. Evaluate Strategic 
Alternatives & Select 
“Bundles” 




4 



Design and Implement Plan for: 



10. Physical Domain / 
Subsystems 

11 . Institutional Sphere 



12. Evaluate, Monitors 
Adapt Strategic Alternatives 
for CLIOS System 


Figure 20. Iteration in the CLIOS Proeess (From; Sussman, 2007, p. 16) 
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IV. CLIOS STAGE 1 REPRESENTATION—MAJOR DEFENSE 

ACQUISTION SYSTEM 


A. INTRODUCTION 

The first stage of the CLIOS process—Representation—is the first order for 
understanding the CLIOS System, in this case the U.S. major defense acquisition system. 
The key functions of this stage are (a) to begin to understand the major defense 
acquisition system as a whole and (b) to establish preliminary goals for this system. The 
outputs for this stage are (1) system description, (2) issue identification, (3) goal 
identification, and (4) a structural representation. 

The CLIOS framework is an iterative process. As an analyst moves through the 
stages and obtains a greater understanding of the system, s/he will return to previous 
stages or steps to modify, refine, and improve. The complex nature of the CLIOS system 
requires this. 

This chapter presents a partial CLIOS application representing the physical 
domain of the U.S. major defense acquisition system nested within an institutional 
sphere. The first four steps of the CLIOS Process, which are found in Stage 1, are 
applied to the major defense acquisition system. 

B, STEP 1—DESCRIBE THE SYSTEM: CHECKLISTS AND 

PRELIMINARY GOAL IDENTIFICATION 

Step 1 is the initial system description and preliminary goal identification. The 
CLIOS Process facilitates system description with a series of checklists for a high-level 
assessment. There are three checklists: Characteristics Checklist, Opportunities / Issues / 
Challenges Checklist, and Preliminary CLIOS System Goals Checklist. Refer back to 
Figure 17 on page 55 for more detail on checklists. These items may range from detailed 
descriptions to high level bullet points. The key idea in Step 1 is to initially frame the 
system. Although much of the literature review in Chapter II covers many of the items 
for the checklists in Step 1, below is a summation of the key points. 
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1 . 


Characteristics Checklist 


The Characteristics Checklist has a series of questions to facilitate the 
development of the checklists. 

a. What is the Temporal and Geographic Scale of the System? 


Characteristics Checklist - 

Temporal Scale; TIME SCALE 

--- > 


Military Real Time Tactical Planning Strategic Planning 

Industry Concept Detail Design Production 


Executive 
Office / 
Congress 


Submit Debate 

Budget 

(for one aspect 
of the system) 


Approve / Reject 


Eigure21. Temporal Scale 


Depending on whose perspective is used (see Eigure21), the time scale for 
the major defense acquisition system may be different. DoD spends three years of 
planning before submitting the request to be added to the President’s budget. For the 
defense industry, from concept to delivery, acquiring a major weapon system could take 
well over 10-15 years depending on the maturity level of technology. 
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Characteristics Checklist - 


Geographical Scale; 

GEOGRAPHIC 

SCALE 



Local Regional National 

Global 


Figure 22. Geographic Scale 


The weapon systems are designed and produced all over the U.S., 
including prime and subcontractors, vendors, program executive offices, stakeholders, 
etc. In addition, the actual weapons systems are utilized on a local, regional, national, 
and global basis (See Figure 22 above). 

b. What are the Core Technologies and Systems? 


Characteristics Checklist continued - 
Some Core Technologies and Systems; 

• Software 

• 

Airborne Easer (ABE) 

• Easer 

• 

Global Hawk Unmanned Aircraft System 

• Radio 

• 

Kinetic Energy Interceptors 

• Physics 

• 

DDG 1000 Destroyer 

• Autonomous 

• 

E-2D Advanced Hawkeye 

• Physics 

• 

Terminal High Altitude Area Defense 


• 

(THAAD) 

And many more... 


Figure 23. Core Technologies and Systems 


By 2007, DoD had 95 major defense acquisition programs in its portfolio 
(GAO, 2008). Major weapon systems include but are not limited to the list above. These 
systems require advanced technology. Years ago, DoD bought hardware, e.g., ships. 
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aircraft, etc., which were eontrolled by the warfighter. Today, the systems are advaneed 
and eomplex with integrated software eontrolling most of the eore subsystems within the 
platform. 


c. What are the Key Economic and Market Issues? 

Industrial base . Nearly 25 years ago, defense programs had over 20 prime 
eontraetors eompeting for multiple programs every year (DAPAP, 2005). Today there 
are four dominant prime eontraetors eompeting for fewer and fewer programs. 
Contraetors have had to reduee plant eapaeity or merge with others. Although it is 
important for eompetition to play a role, fewer prime eontraetors and lower produetions 
mean higher eosts. Also, sinee one prime eontraetor eannot monopolize the market, 
eontraets are usually shared beeause it is neeessary to keep the workforee and industry 
knowledge eurrent with more than one eompany. 

Workforee. Both the workforee produeing the aetual weapon systems and 
the workforee involved as military or eivilians working the aequisition side are erueial to 
the system. Congressmen lobby for eertain programs, whieh are produeed in their states. 
The employment of their eonstituents is a key to eeonomie development in their area. 

Inflation. Many of the GAO reports reviewed in Chapter II eited inflation 
as being a key eoneern for inerease in eost estimates. 

d. Are There any Important Social or Political Issues or 
Controversies that Relate to this System? 

Pork barrel or Earmarks : As mentioned above under workforee, 
eongressmen have an agenda that may be different from the need or the good of the 
whole. It is in their interest and that of their eonstituents to reeeive defense eontraets for 
eompanies in their area or state for eeonomie and/or sustainable development. Therefore, 
there is an ineentive for them to lobby for programs that benefit them but may not benefit 
the nation as a whole. 

Unrequested weapons . Congress is known for adding items to the budget 
that were not neeessarily requested by DoD. For example, this year (2009) Seeretary of 
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Defense Robert Gates deleted the F-22 Raptor Air Dominanee Fighters from the budget. 
However, Congress threatened to add them. 

Immature teehnology . Although there is a poliey for not approving and 
funding projeets until the teehnology has been demonstrated, programs eontinually 
reeeive funding with unproven and immature teehnology, requiring more funds later in 
the proeess and, of eourse, affeeting sehedule and performanee. 

International seeurity environment . The wars in Iraq and Afghanistan 
have demonstrated we are fighting a different kind of enemy that requires different 
strategies and weapons. 

e. What is the Historical Development and Context of the Major 
Defense Acquisition System? 

The U.S. aequisition system has been experieneing problems for more 
than a eentury and has been undergoing reform for more than half a eentury. The 
planning, programming, and budgeting system was instituted in 1962 when Seeretary of 
Defense Robert MeNamara deeided to run the government like a business (Kausal, 2000). 
There has been a long history of inaeeurate estimations, requirements ereep, and poor 
program management. 

f What are the Policy Questions that Need to be Addressed? 

Funding . For stability, these major defense aequisition systems need long¬ 
term eommitment, not year-to-year budget approval. Commitment to funding ereates 
stability for the eontraetors, allowing them to foeus on delivering eapabilities rather than 
gaining ineremental funding. This funding eommitment to programs allows program 
managers to identify unknowns as high risks early without eritieism or loss of funding. 

Teehnology. Although there are polieies in plaee to avoid the risk of 
aequiring major defense systems without mature teehnologies, eountless programs 
eontinue with unproven teehnology. 
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Oversight vs. Accountability . DoD has a complicated acquisition system 
emphasizing oversight of programs rather than emphasizing accountability by the 
contractors. 

Cost estimation . The process for estimating costs must be addressed. 
Whether the estimates are low to not cause “sticker shock” and gain initial approval or 
whether there is not enough knowledge of the program to understand the actual costs, the 
system must be improved. 

2, Opportunities, Issues, and Challenges Checklist 

This part of Step 1 captures aspects of the major defense acquisition system that 
later, in Stage 2, will be used to establish strategic alternatives. Information gathered in 
the Characteristics Checklist will be used as well. 

Cost, Performance, Schedule. Historically, cost and schedule have been an issue 
since with military acquisitions since the Civil War (DAPAP, 2005). 

Technology. The U.S. has been a leader in military technology. However, it 
costs money and time to develop these technologies. 

Cost. There is no standard, consistent and coherent cost tracking system that is 
actually used. 

Agendas . There are conflicting values among the acquisition community. As 
mentioned above, congressmen have an agenda that may be different from the need or the 
good of the whole. Industry would like to sell more programs and be paid to develop 
more advanced technology. The military branches have a desire for new, technologically 
advanced weapon systems. The U.S.’s fundamental strategy has been to deliver lethal 
violence beyond the enemies’ capability to withstand or bear. These advanced systems 
provide that capability. In addition, with the all-volunteer force, U.S. military branches 
are small. Therefore, the U.S. does not want to repeat the Korean or WW2 wars. The 
U.S. forces are currently stretched in Iraq. The advanced weapon systems enabled rapid 
defeat. However, holding on to “victory” nearly broke the U.S. The issues are not well 
resolved between political and military interests. 
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Threats . There are ehanges in the way war is handled-engaging different types of 
enemies requires different weapons and taeties. In the past, there were well-known, well- 
defined threats. Today, there is no single, well-defined threat. 

Funding / Stability . Although many of the programs require long-term 
eommitment, the budgeting proeess is based on short-term deeision making. Long-term 
eost inereases are aeeeptable in order to aehieve short-term budget savings. 

Eeonomie. The major defense aequisition system is an eeonomie engine 
employing hundreds of thousands aeross the nation, yet dependent on the eeonomy. 

Value. In the past, the U.S. has foeused on the eontinuous fielding of systems 
with the best possible performanee, not neeessarily foeused on the best overall value for 
money. 


3. Preliminary Major Defense Acquisition System Goals Checklist 

. Provide the best, neeessary eapability to the warfighter to perform his/her 
job suoeessfully for the best value. 

. Control eost, sehedule, and performanee. 

Step 1 begins to layout the full range of the system. The purpose of Step 1 is to 
highlight the main issues, identify important problems and possible eauses, and provide a 
preliminary bounding of the system. This step helps determine what subsystems and 
eomponents will be ineluded, as well as important link s . This is an iterative proeess. 
Later stages may require returning to stage one due to initially framing the system 
ineorreetly. It is likely important aspeets have been left out or only partially represented. 

C. STEP 2—IDENTIFY SUBSYSTEMS IN PHYSICAL DOMAIN & GROUPS 

ON INSTITUTIONAL SPHERE 

Step 2 of stage one requires the analyst to identify the major subsystems in the 
physieal domain and the major aetor groups on the institutional sphere. The major 
subsystems within the physieal domain ean be teehnieal, natural, eeonomie, soeial, and 
politieal. The major aetor groups on the institutional sphere are the ones who are affeeted 
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by the subsystems within the physical domain, who attempt to influence those 
subsystems, and/or who “worry” about them (Sussman, 2007). 

For the physical domain, one logical approach to identifying subsystems is using 
the ones already defined by DoD; the Planning, Programming, Budgeting, and Execution 
(PPBE) system, the Joint Capabilities Integration and Development System (JCIDS), and 
the DoD 5000.02 Acquisition Process. 

Identification of the subsystems within physical domain; 

• Budget (PPBE) 

. JCIDS 

. Defense Acquisition System 

As stated in Chapter II , the PPBE is how the DoD allocates its resources. “It’s 
how DoD, and their contractors, manage to stay within their fiscal budget while they 
follow the Secretary of Defense’s policy, strategy, and goals” (DAU, 2009). The PPBE 
process overlaps with and supports the acquisition process by providing feedback and 
adjustments as needed during the execution phase. JCIDS key role is identifying required 
capabilities for the warfighters in support of national defense strategies. It is the 
mechanism for requirements generation and supports the Joint Requirements Oversight 
Council (JROC). The JCIDS process supports the acquisition and PPBE processes by 
identifying capability needs and associated performance criteria in order to develop and 
produce the necessary weapon systems. Einally, the defense acquisition system is the 
management process that guides all DoD acquisitions programs, including major defense 
programs. Eigure 24 demonstrates how these three entities interact for major weapon 
systems acquisition. Eor more detail, go the Defense Acquisition University’s website, 
https://acc.dau.mil/ifc/ . 
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Figure 24. Integrated Defense Aequisition (From; DAU website, https://aee.dau.mil/ife/ ) 
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Although the physical domain has previously been identified and mapped by the 
Defense Acquisition University, these subsystems alone do not explain the complexity or 
describe the major defense acquisition system as a whole. There are major actor groups 
that have influence over or are influenced by the subsystems. 

Initial identification of major actor groups on the institutional sphere; 

. Congress 

. Industry 

. DoD 

. Military (Individual branches) 

• Executive Branch 

. U.S. public 

• Security Threats 

Due to the iterative nature of the framework, more major actor groups may be 
identified at other times during the CLIOS Process. 

D. STEP 3—POPULATE THE PHYSICAL DOMAIN & THE 

INSTITUTIONAL SPHERE 

Step 3 requires populating the physical domain and institutional sphere for the 
major defense acquisition system and identifying interactions between the two. See 
Figure 25. 

Each of the subsystems within the physical domain is layered and includes 
influences from the institutional sphere. There is overlap among the PPBE, JCIDS, and 
Acquisition subsystems. 
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Figure 25. Major Defense Aequisition System—Layering of Subsystems and 

Institutional Sphere 


Within the physieal domain, there is overlap and links between the subsystems. 
Inside the institutional sphere, there is mueh interaetion among the major aetor groups. 
Furthermore, there are many links among the major aetor groups in the institutional 
sphere and the major subsystems in the physieal domain. See Figure 26 for a maero-level 
view of the links among the major aetor groups within the institutional sphere and links 
among the major aetor groups and the subsystems in the physieal domain. 
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INSTITUTIONAL SPHERE 



Figure 26. Major Defense Aequisition System—Links Between Institutional Sphere and 

Physical Domain 
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E, STEP 4—DESCRIBE COMPONENTS AND LINKS 

In the final step of the representation stage, Step 4, the analyst deseribes the 
eomponents in the physieal domain, the aetors in the institutional sphere, and the links 
among them. Within the physieal domain, there are three subsystems; PPBE, JCIDS, 
and Defense Aequisition System. These three subsystems were designed to assist senior 
deeision makers within the major defense aequisition system. They internet with eaeh 
other, as well as operate separately. These three systems have been previously defined in 
both Chapter II and earlier in the fourth ehapter. These subsystems have been identified 
by DoD as major deeision-support systems. 

1. Physical Domain—Planning, Programming, Budgeting, and 

Execution (PPBE) 

The PPBE is the system used by the DoD to ereate their portion of the President’s 
budget. This is DoD’s strategie planning, program development, and resouree alloeation 
proeess. PPBE has links to both JCIDS and the aequisition system. It supports JCIDS 
with doeuments and training. It supports the aequisition system by understanding 
Seeurity Threats and providing feedbaek during the exeeution phase. PPBE is a part of 
DoD that works with Military, Congress, and the President and employs U.S. eivilians. 

2. Physical Domain—^Joint Capabilities Integration and Development 
System (JCIDS) 

The Joint Chiefs of Staff, under DoD, ereated JCIDS to identify gaps in military 
needs and generate requirements. It internets elosely with the aequisition system to 
integrate eapabilities in the aequisition system. The system is influeneed by the Military 
and its needs, by Seeurity Threats, and by Industry’s advaneement of teehnology. The 
produet influenees Congress and the President’s deeision eoneerning the need and 
funding for the major defense systems. 
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3. Physical Domain—Defense Acquisition System 


The Defense Aequisition System is the management proeess that guides all DoD 
major defense aequisition programs. It provides the polieies that govern the defense 
aequisition programs. Major defense aequisition programs progress through a series of 
milestones assoeiated with major phases of the program. It guides DoD personnel as well 
as Industry and Military. Congress is updated on the progress of these programs. 

The major aetors within the institutional sphere of the major defense aequisition 
system are Congress, DoD, the Military, the Exeeutive Braneh, Industry, U.S. Publie, and 
Seeurity Threats. 

4. Institutional Sphere—Congress 

Congress has two major responsibilities when working within the major defense 
aequisition system; legislative proeess and the oversight funetion. Without Congress, 
there would be no major defense programs and no money to buy major weapon systems. 
Without major defense programs, there would be no national defense or jobs surrounding 
national defense, whieh ineludes military and eivilians aeross the world. Both the Senate 
and the House have eommittees that influenee DoD and the budgets. The Senate Armed 
Serviees Committee is responsible for a wide variety of poliey and budgetary issues. The 
House Armed Serviees Committee has a wide range of responsibility ineluding researeh 
and development in support of defense systems. This major aetor group also funetions as 
individuals. Eaeh member of Congress is eleeted, and, usually, eaeh has the desire to be 
re-eleeted—a prime motivator. Therefore, members may vote for or against items 
depending on the impaet it has on his/her eonstituent’s opinion of the eongressman—not 
on the impaet of the nation as a whole. Allianees are formed among members of 
Congress to assist eaeh others’ agendas. Congress is influeneed by and influenees 
Seeurity Threats, the DoD, the President, Industry, the U.S. Publie, and the Military. 
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5. 


Institutional Sphere—Department of Defense 


Although DoD is part of the Presidential Cabinet, it is eonsidered a separate major 
aetor group within the institutional sphere. The DoD basieally has dual roles of 
responsibility: warfighting role and aequisition and logisties support role. It is DoD who 
proeures and manages the major defense programs. Both the DoD and the military 
generate requirements for future military needs. All defense programs must be derived 
from doeumented and validated mission needs. For major defense programs, it is the 
Joint Chiefs of Staff who must approve the requirement for the program. However, 
Congress must give its approval for funding of the programs. Like Congress, DoD has 
mueh influenee over aequisition poliey. It is influeneed by Congress, the President, the 
Military, Seeurity Threats, and Industry. 

6, Institutional Sphere—The Military 

The military branehes are not only the end users of the major defense systems, but 
they also play a key role in influeneing the aequisition system. Eaeh of the military 
departments is organized under the Seeretary of Defense. They help with determining 
mission needs and generating requirements for major defense programs. The Military 
works elosely with Industry, giving out funding for researeh of advaneed teehnology and 
overseeing new or existing programs. The Military influenees Industry, edueating them 
on needs and requirements. On the other hand. Industry influenees the Military 
suggesting new or different teehnology they believe would be better for the Military in an 
effort to shape the requirements. The Military antieipates and responds to Seeurity 
Threats. 


7. Institutional Sphere—The Executive Branch 

The President is the Commander-in-Chief of the armed forees. However, it is 
Congress who has the power to deelare war, determines the size of the military, ereates 
poliey that govern the military, and gives approval for funding major defense systems. 
The President submits a budget, whieh ineludes major defense programs, to Congress. 
After mueh diseussion. Congress either approves with ehanges or disapproves. However, 
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the President has the final voiee of vetoing or signing these bills. The Exeeutive Braneh 
works elosely with DoD, sinee DoD is part of the Presidential Cabinet. The President 
appoints the head of the DoD, the Seeretary of Defense. As witnessed earlier this year in 
2009, the Seeretary of Defense has mueh infiuenee over Congress, where Defense 
Seeretary Gates warned Congress of spending money on unneeessary weapons. The 
President baeked the Seeretary of Defense saying he would veto any bill by Congress 
with funding for F-22s—an example of the interaetions among these major aetor groups 
within the institutional sphere. 

8, Institutional Sphere—Industry 

Industry plays a large role in the major defense aequisition system. It is industry 
that designs and produees the major defense weapons, employs hundreds of thousands of 
U.S. eitizens to manufaeture these weapons, and seeks to aequire more defense eontraets 
for profit. Industry spends a portion of its own money in developing new teehnologies to 
shape the requirements of major defense systems. It lobbies to Congress in support of the 
systems they produee. Industry keeps abreast of Seeurity Threats and designs solutions 
for DoD and the Military. 

9. Institutional Sphere—The U.S. Public 

The U.S. publie plays a smaller role in the major defense aequisition system 
eoneerning day-to-day deeisions on requirements, numbers, funding, ete. However, they 
still have infiuenee on the system as a whole. One of the areas of infiuenee is the many 
U.S. eitizens who are employed as part of the defense-eontraeting arena or the eivilian 
aequisition workforee. The employment not only helps them as individuals, but it also 
stimulates eeonomie development in their region. Another area of infiuenee is their right 
to vote. It is the U.S. publie that ehooses both the President and the eongressmen. The 
U.S. publie is infiueneed by Seeurity Threats. Many eitizens are aetively involved in 
trying to infiuenee their eongressman and are eoneerned about either too mueh military 
presenee in other nations, not enough military presenee, anti-war, pro-war, ete. With the 
wars in Iraq and Afghanistan, many families have been affeeted by members serving or 
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dying in these wars and all of the other soeial implications surrounding it. The U.S. 
public has a voice, and therefore influence, on the use of their tax dollars. 

10. Institutional Sphere—Security Threats 

Security threats have always played a key role in the major defense acquisition 
system. The Cold War is a great example of a threat that influenced the acquisition and 
production of multiple major defense systems. It is the goal of the U.S. to protect its 
interests. Thus, when security is threatened, the military is called on and equipped to 
fight the threat. 

F. CLIOS ANALYSIS 

Basically, there are four goals among the major actor group for acquiring major 
weapon systems; (1) national security through acquiring systems with the required 
capability; (2) economic growth as seen with employment, generation of technology, and 
foreign exchange earning thru imports; (3) economic stability by acquiring capable 
systems while keeping public expenditures within limits; and (4) continue to build or 
sustain relations with other nations (Kausal, 1999). Therefore, the question remains as 
follows; Is it possible to (a) buy weapon systems with (b) enhanced capabilities at an (c) 
inexpensive price for financial stability and within a budget during a (d) reasonable 
amount of time while (e) generating jobs and useful technology in both the defense and 
civil sectors, which still (f) sustains or strengthens relations with other nations? 

Past answers to this question, and solutions to this complex problem, have 
involved compromise. Future answers most likely will continue to include compromise 
on one or more of the elements involved. However, with the insight gained through the 
CLIOS Process, a more holistic approach can be taken to identifying the real problems 
with the system and implementing solutions. Although only the Representation Stage has 
been applied to the major defense acquisition system, preliminary analysis can begin 
using the knowledge obtained of the subsystems in the physical domain and the major 
actor groups within the institutional sphere. Obviously, it is necessary to proceed through 
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the iterative CLIOS proeess to understand, validate, refine, or modify. Nevertheless, with 
this eurrent knowledge, two observable problems with eorresponding solutions are 
presented. 


1. Observation I: Long-term Decisions—Short-term Information 

From the CLIOS analysis, one of the problems appears to be long-term deeisions 
being made on short-term information. The problem illustrated in Figure 27 shows long¬ 
term system aequisition deeisions made in an environment utilizing short-term 
information. The result is a tendeney for the institutions below the dashed line to 
underestimate future eosts. 

The literature review eondueted in Chapter II repeatedly revealed funding and 
produetion deeisions being made on programs where realistie teehnology performanee 
was unknown. Garvin and Roberto (2001) said, "faets eome in two varieties: those that 
have been earefully tested and those that have been merely asserted or assumed” (p. 32). 
Due to the time period imposed, there is not enough time to earefully test. 

In their researeh on why people or groups make deeisions without the neeessary 
knowledge, Cho, Jerrell, and Landay (2000) identified seven barriers to knowledge-based 
deeisionmaking. The barriers identified apply to the weapons defense aequisition system: 
(1) Limited understanding of the total eustomer need; (2) Politieal/finaneial instability; 
(3) No time to share insights or solutions; (4) Culture Clash, (5) Proeess/eultural 
eomplexity; (6) Ineffeetive or nonexistent eommunieation; and (7) Geographie dispersion 
of workers (p. 2-7). In order to have knowledge of a partieular subjeet, a person 
undergoes a sequenee of steps. In the beginning, data is gathered from dispersed 
elements. Onee a pattern is seen in the data, then information is obtained. However, a 
person or group does not have knowledge until both the information plus experienee 
eome together. The eurrent eyele for making deeisions, does not allow enough time to 
make these important deeisions based on real knowledge. 
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Figure 27. CLIOS Representation; Long-term Decisions - Short-term Information 

2, Solution I: Integrated Process Team 

A possible strategic alternative to the current operation is an Integrated Process 
Team. Congress, JCIDS and the Defense Acquisition System would form an integrated 
process team jointly considering both near and far-term economic conditions and security 
threats before new long-term defense programs were approved and funded. See Figure 28 
for an illustrated depiction. 
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Figure 28. Integrated Process Team 

Of course, there are barriers and challenges to an idea such as this, one being the 
challenge of implementing such a team. It is important to bring together the right people 
to work together as a team on this problem. The idea of an integrated team is not new. 
France has Integrated Program Teams that includes the individual services, DGA, which 
is the DoD equivalent, and industry. Germany has Integrated Project Teams, which bring 
together all stakeholders including industry except during competition. The UK has 
Integrated Project Teams where responsibility and accountability are combined to focus 
on the customer and achieve seamless flow of responsibility from concept to disposal. 
Like Germany, the UK involves all stakeholders including industry except during 
competition. The U.S. uses many Integrated Product Teams, each focusing on a different 
product. 
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3. 


Solution II: Mandated Systems Engineering Organization 


Another possible strategic alternative is to have a new federally mandated systems 
engineering organization. This systems engineering organization would consider 
economic and threat conditions, analyze technological maturity, and make independent 
cost estimates as inputs to new defense system acquisition decisions. It would be equally 
accountable to Congress and DoD, not just a DoD office as the one stood up in early 
2009. If the technology is immature, the federal systems engineering organization would 
determine requirements for technology development programs. Industry would be 
awarded contracts, not on the basis of lowest cost, but rather on the ability to meet the 
federal systems engineering requirements. This alternative does not require formation of 
an integrated team. 



Figure 29. Mandated Federal Systems Engineering Organization 


71 














4. 


Observation II: Working in Silos—Little Accountability 


Another observation is there are times when major aetor groups make deeisions or 
work in silos. Eaeh of the major aetor groups is not “evil.” They work independently for 
eauses, whieh, essentially, are not ineongruous. However, when programs break down, 
fingers are pointed and blame is assigned, while it is hard to determine what went wrong 
and where. These independent aetions faeilitate ineonsisteney throughout the life eyele 
of the program. This results in little aeeountability for the overall problems of the whole 
system. See Figure 30 for a depietion of this. 
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Nature of system has major actor groups working 
independently 

I 


Time over the Life Cycle 


I "■ 




When programs 
have trouble with 
schedule, costs, 
and/or 

performance, it is 
difficult to 
determine the 
source. 


Nature of system has little accountability 


► 

Further along it is easy to blame 
industry, but problems may have 
been instigated from decisions made 
by other actor groups 


Figure 30. CLIOS Representation; Working in Silos - Little Aeeountability 


5, Solution III: Defense Acquisition Accountability 


This final possible strategie alternative is to have a hybrid of the Integrated 
Proeess Team and the Federal Systems Engineering organization. This hybrid would be a 
program-based eongressional, DoD and industry eoalition resulting in eaeh partner 
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economically penalized for large overruns, schedule slips, or other issues within a 
program. All will be held accountable by the independent. Federal Systems Engineering 
organization. See Figure 31. 


Accountability would have to be instilled at every level in order for consistency 
and stability to thrive within the program. Changes will include concrete requirements, 
full commitment to funding, and minimized time from order to delivery. 



Figure 31. Defense Acquisition Accountability 
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G. 


CONCLUSION 


The first stage of the CLIOS Proeess, the Representation Stage, was applied to the 
major defense aequisition system. Essentially, there are four steps within this stage. 
Eaeh step was diseussed eoneerning the major defense aequisition system. The outputs 
of this stage rendered a better understanding of the subsystems within the whole system 
as well as the major aetor groups influeneing the system. As this system eontinues to 
progress through the other stages of the CEIOS Proeess, the iterative nature of the 
proeess will require returning to the Representation Stage and modifying or refining as 
neeessary. 

Despite eontinuing with the CEIOS Proeess, initial analysis was eondueted based 
on the observations of Stage 1. This resulted in two observations: (1) Eong-term 
deeisions made on short-term information and (2) Independent working leads to lesser 
aeeountability. Three strategie alternatives were identified: (1) form an Integrated 
Proeess Team with Congress, JCIDS and the Defense Aequisition System to jointly 
eonsider both near and far-term eeonomie eonditions and seeurity threats before new 
long-term defense programs were approved and funded; (2) form a mandated Federal 
Systems Engineering organization equally aeeountable to both Congress and DoD to 
eonsider eeonomie and threat eonditions, analyze teehnologieal maturity, and make 
independent eost estimates as inputs to new defense system aequisition deeisions; and (3) 
ereate a hybrid of the two previous solutions to implement aeeountability at all levels of 
the aequisition to all major aetor groups. Of eourse, ehallenges to implementation will 
exist. However, as stated above, eontinuing with the CEIOS Proeess will faeilitate in 
refining the strategie alternatives, as well as implementing them. 
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V. CONCLUSION 


A, CONCLUSION 

Analyzing the major defense aequisition system is a ehallenging, eomplex 
proeess. The system has been plagued with problems sinee the beginning. Hundreds of 
reports were written on these issues and reeommendations for improvement have been 
plentiful. Although soeial seienee researeh sueh as this usually eoneludes with direet 
reeommendations for ehanging publie poliey, this researeh is limited to providing a better 
foundation for understanding the system. The goal of this researeh was to improve 
understanding of the intrieaeies of the U.S. major defense aequisition system for framing 
future solutions. The CLIOS Proeess provides sueh a framework for understanding a 
eomplex, large-seale, intereonneeted, open, soeioteehnieal system. 

B, SUMMARY OF KEY POINTS 

The system surrounding major weapons aequisitions eontinues to produee 
unwanted results: enormous eost overruns, unsatisfaetory sehedule delays, and deereased 
performanee. The U.S. major defense aequisition system is very eomplex with many 
stakeholders, deeision makers, and outside influenees. The major defense aequisition 
system is not influenoed by DoD and Congress only. It is subjeet to the President’s 
approval, the requirements of the military, the eapability of industry, the seeurity threats 
from other groups or states, and the opinions of the Ameriean publie. 

Although polieies are in plaee to thwart the progression of a program with 
unproven teehnology. Congress eontinues to approve budget for these programs with 
undemonstrated teehnology resulting in inereased eosts, sehedule delays, and lesser 
performanee than antieipated. Congress adds to the issues by not eommitting long-term 
to programs that do have proven teehnology. The funding is released inerementally 
without assuranee for the future. Congress eontinues to add items to the DoD budget, 
whieh were never requested by DoD. Yet, it slashes other programs, whieh were 
requested based on identified mission needs and requirements. On the other hand, DoD 

77 



continues to ask for weapons systems for wars that are no longer being fought, not 
reeognizing the U.S. has a different enemy to fight with different eapabilities needed to 
be sueeessful. 

Industry, along with other assoeiated aequisition personnel, is eontinually blamed 
for insuffieient management or mismanagement in general. There are very few prime 
eontraetors eompeting for very few major defense weapons aequisition programs. In 
order to keep eontraetors “alive” and to keep a sizeable and knowledgeable defense 
workforee, many eontraets are split and shared between two or three. 

C. LESSONS LEARNED / HEURISTICS 

There are hundreds of quality, valuable reports available to better understand the 
system and influenees. GAO eontinues to report on major defense aequisitions and 
eontinues to make reeommendations. However, it appears that either no one takes heed, 
or polieies are ereated but not implemented. 

What is best for the nation is not always the agenda of the stakeholders. 
Congressmen eontinue to lobby for programs that are best for their region and 
eonstituents but not neeessarily best for the eurrent U.S. military needs. The military 
eontinue to ask for more than originally required or funded eausing requirements ereep. 
Industry shapes the military requirements based on their produets. 

The best teehnology has beeome the enemy of good enough teehnology. 
Advaneed, unproven teehnology has been an issue with most major defense weapon 
system aequisitions. The U.S. is the dominant military in the world with the most 
advaneed teehnology. The U.S. spends more than any other eountry. Some of the 
advaneed teehnology being sought is not neeessary to fight and win against the threats 
faeed by the U.S. today. 

Aeeountability must be injeeted in the aequisition eulture. There are “too many 
hands in the pot” to know who is aeeountable for what. In terms of the CLIOS Proeess, 
there are too many major aetor groups with influenees on the system. Everyone involved 
in the aequisition proeess—Congress, DoD, the Military, Industry—must assume a sense 
of aeeountability to the U.S. 
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The discipline of systems engineering has already defined guidelines for program 
success; Requirements management and definition; Systems architecture development; 
System, subsystem design; Systems integration and design; Validation and verification; 
System development and post deployment; etc. These concepts should be practiced by 
all stakeholders, and major actor groups, within the major defense acquisition system. 
Although DoDD 5000.02 mandates systems engineering on programs, it seems to apply 
to the industry contractors. Systems’ thinking is not prominent in all aspects of the 
system. Everyone involved in decision making concerning major defense acquisition 
must understand systems engineering and must employ a systems thinking approach. 

The CLIOS Process unveils the complexity of the system. Utilizing the CLIOS 
framework emphasizes the complexity of the U.S. major defense acquisition system even 
more. The Representation Stage is just the beginning of wading through the subsystems 
and actor groups. Although applying the entire CLIOS Process to the U.S. major defense 
acquisition system will take time, it will uncover more points of leverage for 
improvement and allow for real changes to be implemented. 

D. RECOMMENDATIONS AND FURTHER RESEARCH 

Further research should be conducted on the U.S. major defense acquisition 
system using the CLIOS Process. The remaining stages of the CLIOS Process, especially 
the iterative aspects of the process, should be applied to the U.S. major defense 
acquisition system. Much more will be discovered about the system, its behavior, and the 
leverage points for strategic alternatives. As the application of the CLIOS Process 
continues, deeper understanding of the system will be obtained. Stage 1 allowed for the 
initial understanding of the system, as well as provided the initial mental mapping of 
links within the physical and institutional systems. This stage produced general insights 
into the U.S. defense acquisition system. However moving into Stage 2, the analyst will 
gain a detailed and quantitative understanding of the system. Stage 2 will provide greater 
understanding and appreciation for system possibilities, limits, uncertainties, and 
sensitivities. Because of the newer comprehension during Stage 2, the analyst will go 
back to Stage 1 and modify and refine the initial structure—an iterative process. Finally, 
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implementing strategie alternatives in Stage 3 will eause ehanges within the physieal 
domain and the institutional sphere surrounding the major defense aequisition system. 
These changes must be evaluated, monitored, and adapted, as needed for improvement to 
the system as a whole. 


80 



LIST OF REFERENCES 


Acquisition History Project Working Paper #3. The Evolution of DoD Directive 5000.1: 
Acquisition Management Policy 1971 to 2003. Retrieved April 22, 2009 from 
http://www.historv.armv.mil/aequisition/researeh/working3.html 

Cho, G., Jerrell, H., & Landay, W. (2000). Program management 2000: Know the way. 
Fort Belvoir, VA: Defense Systems Management College Press. 

CIA World Faetbook (2009). Retrieved July 31, 2009 from 

https://www.eia.gov/librarv/publieations/the-world-faetbook/index.html 

Congressional Researeh Serviee (2008). CRS Report to Congress—Defense Acquisitions: 
Overview, Issues, and Options for Congress [Eleetronie version]. Washington, 
D.C.: U.S. Government Printing Offiee. 

Defense Aequisition Performanee Assessment Projeet (2005). Defense Acquisition 
Performance Assessment: Executive Study [Eleetronie version]. Washington, 
D.C.: U.S. Government Printing Offiee. 

DefenseEink. (2009). Robert S. MeNamara. Retrieved August 1, 2009 from 

http://www.defenselirLk.mil/specials/seedef histories/bios/mcnamara.htm 

Department of Defense Instruetion Number 5000.02 (2008). Retrieved June 5, 2009 
from https://dap.dau.mil/poliev/Pages/overview.aspx . 

Garvin, D. A. & Roberto, M. A. (2001). What you don’t know about making deeisions. 
Harvard Business Review, Vol. 79 (No. 8). 

Holbrook, M. A. (2003). An Analysis of the Implementation of Acquisition Reform 
Initiatives and Contract Cost Variance. MS Thesis, AFIT/GCA/ENV/03-03. 
Graduate Sehool of Engineering and Management. Wright-Patterson AFB, OH: 
Air Eoree Institute of Teehnology. 

Jones, W. D. (1999). Arming the Eagle: A History ofU. S. Weapons Acquisition Since 
1775 [Eleetronie version]. Washington, D.C.: U.S. Government Printing Offiee. 

Eibrary of Congress. S.454. Retrieved August 1, 2009 from http://thomas.loe.gov/egi- 
bin/bdquerv/z?dl 11 :SN00454:@@@D&summ2=m& 

Kausal, B. A. (Ed.). (1999). A comparison of the defense acquisition systems of France, 
Great Britain, Germany, and the United States. Eort Belvoir, VA: Defense 
Systems Management College Press. 


81 








Kausal, B. A. (Ed.). (2000). A comparison of the defense acquisition systems of Australia, 
Japan, South Korea, Singapore, and the United States. Fort Belvoir, VA; Defense 
Systems Management College Press. 

Maier, M. W. & Rechtin, E. (2002). The art of systems architecting (2"‘^ ed). New York: 
CRC Press. 

Marshal, A. W. & Meckling, W. H. (1959). Predictability of the Costs, Time, and Success 
of Development [Electronic version]. Santa Monica, CA: RAND. 

McKinney, E., Gholz, E., & Sapolsky, H. M. (1994). Acquisition Reform—Lean 94-03. 
Cambridge, MA: Massachusetts Institute of Technology. 

Perry, R., Smith, G. K., Harman, A. J., & Henrichsen, S. (1971). System Acquisition 
Strategies [Electronic version]. Santa Monica, CA: RAND. 

Stockholm International Peace Research Institute (2008). SIPRI Yearbook 2008. 

Sussman, J.M. (2007). The CEIOS process: A user’s guide. Course materials for 

ESD. 04J Frameworks and Models in Engineering Systems, Spring 2007. MIT 
OpenCourseWare, Massachusetts Institute of Technology. Retrieved April 2, 
2009 from http://ocw.mit.edu . 

United States General Accounting Office (1969, June). Statement of Elmer B. Staats, 

Comptroller General of the United States, before the Subcommittee on Economy 
in Government, Joint Economic Committee on the Military Budget and National 
Economic Priorities [Electronic version]. Washington, D.C.: U.S. Government 
Printing Office. 

United States General Accounting Office (1969, November). Statement of Elmer B. 

Staats, Comptroller General of the United States, before the Subcommittee on 
Economy in Government, Joint Economic Committee on the Military Budget and 
National Economic Priorities [Electronic version]. Washington, D.C.: U.S. 
Government Printing Office. 

United States General Accounting Office (1970). Status of the Acquisition of Selected 

Major Weapon Systems [Electronic version]. Washington, D.C.: U.S. Government 
Printing Office. 

United States General Accounting Office (1971, March). Acquisition of Major Weapon 
Systems [Electronic version]. Washington, D.C.: U.S. Government Printing 
Office. 

United States General Accounting Office (1971, November). Comparison of Estimated 
Cost Data [Electronic version]. Washington, D.C.: U.S. Government Printing 
Office. 


82 



United States General Aeeounting Offiee (1972). The Importance of Testing and 

Evaluation in the Acquisition Process for Major Weapon Systems [Eleetronie 
version], Washington, D.C.: U.S. Government Printing Offiee. 

United States General Aeeounting Offiee (1973). Financial Status of Selected Major 
Weapon Systems [Eleetronie version], Washington, D.C.: U.S. Government 
Printing Offiee. 

United States General Aeeounting Offiee (1974). Financial Status of Selected Major 
Weapon Systems [Eleetronie version], Washington, D.C.: U.S. Government 
Printing Offiee. 

United States General Aeeounting Offiee (1975). Financial Status of Selected Major 
Weapon Systems [Eleetronie version], Washington, D.C.: U.S. Government 
Printing Offiee. 

United States General Aeeounting Offiee (1976). Financial Status of Major Acquisitions, 
June 30, 1975 [Eleetronie version], Washington, D.C.: U.S. Government Printing 
Offiee. 

United States General Aeeounting Offiee (1977). Financial Status of Major Acquisitions, 
June 30, 1976 [Eleetronie version], Washington, D.C.: U.S. Government Printing 
Offiee. 

United States General Aeeounting Offiee (1979, January). Financial Status of Major 
Federal Acquisitions, September 30, 1978 [Eleetronie version], Washington, 

D.C.; U.S. Government Printing Offiee. 

United States General Aeeounting Offiee (1979, June). Statement of Jerome H. 

Stolarowe, Director, Procurement and Systems Acquisition Division [Eleetronie 
version], Washington, D.C.: U.S. Government Printing Offiee. 

United States General Aeeounting Offiee (1980). Issues Identified in 21 Recently 

Published Major Weapon System Reports [Eleetronie version], Washington, D.C.: 
U.S. Government Printing Offiee. 

United States General Aeeounting Offiee (1982). Improving Effectiveness and 

Acquisition Management of Selected Weapon Systems: A Summary of Major 
Issues and Recommended Actions [Eleetronie version], Washington, D.C.: U.S. 
Government Printing Offiee. 

United States General Aeeounting Offiee (1983). Weapon Systems Overview: A Summary 
of Recent GAO Reports, Observations, and Recommendations on Major Weapon 
Systems [Eleetronie version], Washington, D.C.: U.S. Government Printing 
Offiee. 


83 



United States General Aeeounting Offiee (1984). Statement of Charles A. Bowsher, 
Comptroller General of the United States, before the Committee on General 
Affairs, United States Senate, on Need for Improved Cost Estimating on Major 
Weapon Systems and Credible Reporting of Planned and Actual Costs [Eleetronie 
version], Washington, D.C.: U.S. Government Printing Offiee. 

United States General Aeeounting Offiee (1988). Major Acquisitions Summary of 
Recurring Problems and Systemic Issues: 1960-1987 [Eleetronie version]. 
Washington, D.C.: U.S. Government Printing Offiee. 

United States General Aeeounting Offiee (1992). Defense Weapons Systems Acquisition 
[Eleetronie version]. Washington, D.C.; U.S. Government Printing Offiee. 

United States General Aeeounting Offiee (1997). Weapons Acquisition: Better Use of 
Limited DoD Acquisition Funding Would Reduce Costs [Eleetronie version], 
Washington, D.C.: U.S. Government Printing Offiee. 

United States General Aeeounting Offiee (1998). Best Practices: Successful Application 
to Weapon Acquisitions Requires Changes in DoD’s Environment [Eleetronie 
version], Washington, D.C.: U.S. Government Printing Offiee. 

United States Government Aeeountability Offiee (2005). DoD Acquisition Outcomes: A 
Case for Change [Eleetronie version], Washington, D.C.; U.S. Government 
Printing Offiee. 

United States Government Aeeountability Offiee (2006). Defense Acquisitions: Major 
Weapon Systems Continue to Experience Cost and Schedule Problems Under 
DoD’s Revised Policy [Eleetronie version], Washington, D.C.: U.S. Government 
Printing Offiee. 

United States Government Aeeountability Offiee (2008). Defense Acquisitions: 

Assessments of Selected Weapon Programs [Eleetronie version], Washington, 
D.C.: U.S. Government Printing Offiee. 

United States Government Aeeountability Offiee (2009). Charting a course for lasting 
reform [Eleetronie version], Washington, D.C.; U.S. Government Printing Offiee. 


84 



INITIAL DISTRIBUTION LIST 


1. Defense Teehnieal Information Center 
Ft. Belvoir, Virginia 

2. Dudley Knox Library 
Naval Postgraduate Sehool 
Monterey, California 

3. John Osmundson, Ph.D. 

Naval Postgraduate Sehool 
Monterey, California 

4. Robert Rifley 

Northrop Grumman Shipbuilding—Gulf Coast 
Paseagoula, Mississippi 

5. Charles Wilson 

Northrop Grumman Shipbuilding—Gulf Coast 
Paseagoula, Mississippi 


85 



